Image encoding device, image decoding device, and program thereof

ABSTRACT

An image encoding device  1  of the present disclosure includes a neighboring pixel non-reference predictor  11  configured to generate a predicted image by a predetermined neighboring pixel non-reference prediction for each pixel signal of an original image in a block unit, a filter processor  12  configured to perform a low-pass filter process on a prediction signal located at a boundary of a block of the predicted image by using a decoded neighboring signal neighboring to the predicted image under a predetermined control, a prediction residual signal generator  55  configured to generate a prediction residual signal of the block unit by using the predicted image, an orthogonal transformer  14  configured to perform an orthogonal transformation process on the prediction residual signal of the block unit under the predetermined control, and an orthogonal transformation selection controller  25  configured to control the filter processor  12  and the orthogonal transformer  14  and generate a predetermined transformation type identification signal. An image decoding device  5  of the present disclosure performs a decoding process based on a transformation type identification signal at the time of the predetermined neighboring pixel non-reference prediction.

RELATED APPLICATIONS

This application is a continuation of PCT/JP2016/074291 filed on Aug.19, 2016, which claims priority to Japanese Application Nos.2015-163260, 2015-163257 and 2015-163259, all filed on Aug. 20, 2015.The entire contents of these applications are incorporated herein byreference.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present disclosure relates to an image encoding device, an imagedecoding device, and a program thereof, which are applicable to a videoencoding method such as MPEG-2, AVC/H.264, MPEG-H, HEVC/H.265, or thelike.

2. Description of the Related Art

Generally, it is known that an image signal has a high signalcorrelation between neighboring pixels, regardless of a still image anda moving image. By using this property, for example, two predictionmodes of an intra prediction and an inter-prediction are prepared in avideo encoding method such as MPEG-2, AVC/H.264, MPEG-H, or HEVC/H.265.The intra prediction is a technique for performing signal predictiononly by signals in an encoded frame, and performs a DC prediction, aPlaner prediction, or a directional prediction with respect to pixelsignals (original signals) in an encoding target block of an originalimage by using pixel signals of encoded and decoded blocks neighboringto a left side or an upper side of the encoding target block, andgenerates a block of a predicted image including pixel signals(prediction signals) predicted by extrapolation. In this way, the pixelsignals in the encoding target block are efficiently predicted by usingthe pixel signals of the decoded block neighboring to the encodingtarget block.

For example, a property of a prediction residual signal by the intraprediction will be described with reference to FIG. 29 by using, as anexample, a prediction (horizontal prediction) in a horizontal directionof the intra prediction. In the example illustrated in FIG. 29, theprediction is performed on an original signal in an encoding targetblock Bko of an original image including pixel signals p having a blocksize of 4 (horizontal)×4 (vertical) pixels (hereinafter simply referredto as “4×4” and the same applies to other block sizes) (see FIG. 29A) byusing a pixel signal of a left-neighboring decoded block as a referencesignal Sr in order for the horizontal prediction, and generates a blockBkp of a predicted image including a prediction signal corresponding tothe pixel signal of the encoding target block (see FIG. 29B).

Then, a block Bkd of the prediction residual signal can be obtained froma difference between the original signal in the encoding target blockBko of the original image and the prediction signal of the block Bkp ofthe predicted image (see FIG. 29C). In the prediction residual signal bythe intra prediction, when a pixel position approaches the referencesignal, prediction efficiency increases and a residual componentdecreases (signal intensity decreases), in other words, when the pixelposition goes away from the reference signal, the prediction efficiencydecreases and the residual component increases (signal intensityincreases) (see FIG. 29D).

Here, in the example illustrated in FIG. 29, the prediction is performedby using inter-pixel correlation in the horizontal direction, and thesignal of the encoded and decoded block also exists above the encodingtarget block. In the example illustrated in FIG. 29, although theprediction is efficiently performed by using the correlation in thehorizontal direction by the intra prediction in the horizontal directionto generate the prediction residual signal, the correlation in avertical direction is not used.

In this regard, in the currently defined H.265, by using the fact thatthe correlation with the pixel signals in the block above the encodingtarget block is high, a filter process that reflects a signal change forthe upper block is applied only to an uppermost prediction signal in theblock Bkp of the predicted image in order to improve accuracy of theprediction signal. With this configuration, not only the correlation inthe horizontal direction by the prediction in the horizontal directionbut also the correlation in the vertical direction can be used, and anaverage residual component in the block Bkd of the prediction residualsignal further decreases and encoding efficiency is improved. Similarly,in the case of an intra prediction in the vertical direction, a filterprocess is applied by using neighboring pixels in the horizontaldirection that are not used for that prediction, and in the DCprediction that does not use horizontal and vertical neighboring pixels,the filter process using the horizontal and vertical neighboring pixelsis applied, so that the encoding efficiency is greatly improved ascompared with the previous video encoding method such as MPEG-2 andH.264.

In addition, in the inter-prediction, a motion prediction is performedfrom a temporally close encoded and decoded reference frame to calculatea motion vector, and a predicted image is generated by using the motionvector. A difference between the predicted image and the original imageis generated as the prediction residual signal. In the conventionaltechnique, however, height of correlation of a signal with a neighboringblock is positively used in the intra prediction, but a propertyindicating the height of the correlation of the signal with theneighboring block is not used in the inter-prediction.

Then, the block of the prediction residual signal generated through theintra prediction or the inter-prediction is subjected to an orthogonaltransformation process and a quantization process and is encoded.

As a feature of a motion compensation process in the inter-prediction, aprediction error amount is statistically large at an end point of aprediction region related to the encoding (that is, the predictionresidual signal for the pixel signal located in an outer periphery ofthe block of the predicted image) (see, for example, Non patentLiterature 1).

CITATION LIST Non-Patent Literature

-   Non-patent Literature 1: Chung Fung Wa et al., “Characteristic    Analysis of Motion Compensation Interframe Differential Signal Based    on Statistical Motion Distribution Model”, D-II, Vol. J84-D-II, No.    9, pp. 2001-2010, Sep. 1, 2001.

SUMMARY OF THE INVENTION

As disclosed in Non patent Literature 1, as the feature of the motioncompensation process in the inter-prediction, the prediction erroramount is statistically large at the end point of the prediction regionrelated to the encoding (that is, the prediction residual signal for thepixel signal located in the outer periphery of the predicted imageblock).

Meanwhile, in the conventional technique, despite the fact that theheight of the correlation of the neighboring signals is positively usedin the intra prediction, such height of the correlation of theneighboring signals is not used in the inter-prediction when theprediction residual signal is generated. Therefore, there is room forfurther improvement of the conventional technique from a viewpoint ofencoding efficiency.

An object of the present disclosure is to provide an image encodingdevice, an image decoding device, and a program thereof, which improvethe encoding efficiency in view of the above problem.

According to the present disclosure, in a signal prediction (referred toas a “neighboring pixel non-reference prediction” in the presentspecification) such as an inter-prediction, in which a signal predictionis performed without using a decoded neighboring signal, by using anencoded and decoded signal (the decoded neighboring signal) neighboringto a block of a predicted image among pixel signals (prediction signals)in the block of the predicted image generated by the neighboring pixelnon-reference prediction, a low-pass filter process is performed on theprediction signal to generate a predicted image including a newprediction signal, based on a predetermined evaluation criterion, andcontrol is performed so that an orthogonal conversion type to be appliedis determined. In particular, when the low-pass filter process isperformed on the prediction signal located at a boundary of the block ofthe predicted image generated by the neighboring pixel non-referenceprediction, the control is performed so that an orthogonaltransformation process in which an end of a transformation base isclosed is performed.

Therefore, even at the time of the neighboring pixel non-referenceprediction, encoding efficiency is improved by making a pixel value(prediction signal) located at a block boundary of a predicted imagebelonging to an end point of the prediction region closer to a pixelvalue of a neighboring block by the low-pass filter process, based onthe predetermined evaluation criterion. Furthermore, by adopting aconfiguration in which the orthogonal transformation process (forexample, DST) in which the end of the transformation base is closed isused for a block of a prediction residual signal obtained based on theblock of the predicted image after the low-pass filter process, theconfiguration matches the feature of the transformation basecorresponding not only to a phase but also to the end point of theprediction region, and the encoding efficiency is more remarkablyimproved.

That is, an image encoding device of the present disclosure is an imageencoding device for encoding by block-dividing an original image of aframe unit constituting a moving image. The image encoding devicecomprises: neighboring pixel non-reference prediction means forgenerating a predicted image of a block unit including a predictionsignal by a predetermined neighboring pixel non-reference predictionthat performs a signal prediction for each pixel signal of an originalimage of the block unit without using a decoded neighboring signal;filter processing means for controlling execution and non-execution of alow-pass filter process under a predetermined selection control, and atthe time of the execution, generating a block of a new predicted imageby performing the low-pass filter process on a prediction signal locatedat a boundary of a block of the predicted image by using the decodedneighboring signal neighboring to the block of the predicted image;prediction residual signal generating means for calculating an error ofeach prediction signal of the block of the predicted image generated bythe filter processing means for each pixel signal of an encoding targetblock of the original image and generating a prediction residual signalof the block unit; orthogonal transformation means for selectivelycontrolling one of a plurality of types of orthogonal transformationprocesses under the predetermined selection control, and performing anorthogonal transformation process on the prediction residual signal ofthe block unit to generate a transformation coefficient signal by theselectively controlled orthogonal transformation process; and orthogonaltransformation selection control means for selectively controlling oneof a first combination for applying a first orthogonal transformationprocess in which an end of a transformation base is closed among theplurality of types of orthogonal transformation processes according tothe execution of the low-pass filter process and a second combinationfor applying a second orthogonal transformation process in which the endof the transformation base is opened among the plurality of types oforthogonal transformation processes according to the non-execution ofthe low-pass filter process, based on a predetermined evaluationcriterion at the time of the neighboring pixel non-reference predictionas the predetermined selection control, and generating, as an encodingparameter, a transformation type identification signal indicating a typeof the selected orthogonal transformation process.

Moreover, in the image encoding device of the present disclosure, thepredetermined neighboring pixel non-reference prediction comprises anyone of an inter-prediction, an intra block copy prediction, and a crosscomponent signal prediction.

[Moreover, in the image encoding device of the present disclosure, theorthogonal transformation selection control means comprises means forselecting the first combination when an effect of the low-pass filterprocess is equal to or higher than a predetermined level or excellent asan RD cost through the execution of the low-pass filter process, as thepredetermined evaluation criterion.

Moreover, in the image encoding device of the present disclosure, thefilter processing means performs the low-pass filter process such that aprediction signal neighboring to the decoded neighboring signal issmoothed among prediction signals in the block of the predicted imagegenerated by the neighboring pixel non-reference prediction means.

Furthermore, an image decoding device of the present disclosure is animage decoding device for decoding a signal encoded by block-dividing aframe constituting a moving image. The image decoding device comprises:inverse orthogonal transformation selection control means for acquiringa transformation type identification signal indicating a type of anorthogonal transformation process selected on an encoding side among aplurality of types of orthogonal transformation processes, andselectively determining a type of corresponding inverse orthogonaltransformation process based on the transformation type identificationsignal at the time of a predetermined neighboring pixel non-referenceprediction that performs a signal prediction without using a decodedneighboring signal for each pixel signal of a block unit; inversequantization means for reconstructing a transformation coefficientsignal by performing a corresponding inverse quantization process on atransformation coefficient quantized by a predetermined quantizationprocess as a signal encoded by block-dividing a frame constituting amoving image; inverse orthogonal transformation means for reconstructinga prediction residual signal of the block unit by performing the inverseorthogonal transformation process determined based on the transformationtype identification signal with respect to the reconstructedtransformation coefficient signal; neighboring pixel non-referenceprediction means for generating a predicted image of the block unitincluding a prediction signal by the neighboring pixel non-referenceprediction; and filter processing means for generating a block of a newpredicted image by performing a low-pass filter process on a predictionsignal located at a boundary of a block of the predicted image by usingthe decoded neighboring signal neighboring to the block of the predictedimage when a first orthogonal transformation process in which an end ofa transformation base is closed by the transformation typeidentification signal.

Moreover, in the image decoding device of the present disclosure, theinverse orthogonal transformation selection control means comprisesmeans for acquiring the transformation type identification signal as anencoding parameter indicating that one of the first orthogonaltransformation process in which the end of the transformation base isclosed and a second orthogonal transformation process in which the endof the transformation base is opened among the plurality of types oforthogonal transformation processes is applied to the encoding side.

Furthermore, a program of the present disclosure is a program forcausing a computer to function as the image encoding device of thepresent disclosure or the image decoding device of the presentdisclosure.

According to the present disclosure, since a signal error occurringbetween a prediction signal of a predicted image and a neighboringdecoded block signal is reduced and encoding efficiency is improved, itis possible to achieve an image encoding device and an image decodingdevice using a video encoding method with high encoding efficiency. Thatis, on an encoding side, a residual component in a prediction residualsignal can be decreased, and the encoding efficiency can be improved byreducing an amount of information to be encoded and transmitted. Also,on a decoding side, it is possible to perform decoding even with such areduced amount of information.

In particular, even at the time of a neighboring pixel non-referenceprediction, the encoding efficiency is improved by making a pixel value(prediction signal) located at a block boundary of a predicted imagebelonging to an end point of the prediction region closer to a pixelvalue of a neighboring block by a low-pass filter process, based on apredetermined evaluation criterion. Furthermore, by adopting aconfiguration in which an orthogonal transformation process (forexample, DST) in which an end of a transformation base is closed is usedfor a block of the prediction residual signal obtained based on theblock of the predicted image, the configuration matches a feature of thetransformation base corresponding not only to a phase but also to theend point of the prediction region, and the encoding efficiency can bemore remarkably improved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram of a periphery of a filter processor relatedto a predicted image in an image encoding device according to a firstembodiment of the present disclosure, and FIG. 1B is an explanatorydiagram illustrating an example of a filter process of the predictedimage.

FIGS. 2A, 2B, and 2C are explanatory diagrams exemplifying a neighboringpixel non-reference prediction according to the present disclosure.

FIG. 3 is a block diagram illustrating an example of an image encodingdevice according to the first embodiment of the present disclosure.

FIG. 4 is a flowchart of the filter process of the predicted image inthe image encoding device according to the first embodiment of thepresent disclosure.

FIG. 5 is an explanatory diagram of the filter process of the predictedimage in the image encoding device according to the first embodiment ofthe present disclosure.

FIG. 6 is a block diagram of the periphery of the filter processorrelated to the predicted image in the image decoding device according tothe first embodiment of the present disclosure.

FIG. 7 is a block diagram illustrating an example of the image decodingdevice according to the first embodiment of the present disclosure.

FIG. 8 is a block diagram of a filter processor in an image encodingdevice or an image decoding device according to a second embodiment ofthe present disclosure.

FIG. 9 is a flowchart of a filter process of a predicted image in theimage encoding device or the image decoding device according to thesecond embodiment of the present disclosure.

FIGS. 10A and 10B are explanatory diagrams of the filter process of thepredicted image in the image encoding device or the image decodingdevice according to the second embodiment of the present disclosure.

FIG. 11A is a block diagram of a periphery of a filter processor and anorthogonal transformation selection controller in an image encodingdevice according to a third embodiment of the present disclosure, andFIGS. 11B and 11C are diagrams illustrating an example of a basewaveform of orthogonal transformation.

FIG. 12 is a block diagram illustrating a first example of the imageencoding device according to the third embodiment of the presentdisclosure.

FIG. 13 is a flowchart of an orthogonal transformation selection controlprocess of the first example in the image encoding device of the thirdembodiment according to the present disclosure.

FIG. 14 is a block diagram illustrating a second example of the imageencoding device according to the third embodiment of the presentdisclosure.

FIG. 15 is a flowchart of an orthogonal transformation selection controlprocess of the second example in the image encoding device of the thirdembodiment according to the present disclosure.

FIG. 16 is a block diagram of the periphery of the filter processor andan inverse orthogonal transformation selection controller in the imagedecoding device according to the third embodiment of the presentdisclosure.

FIG. 17 is a block diagram illustrating an example in the image decodingdevice according to the third embodiment of the present disclosure.

FIG. 18 is a flowchart of an inverse orthogonal transformation selectioncontrol process of an example in the image decoding device according tothe third embodiment of the present disclosure.

FIG. 19A is a block diagram of a periphery of a filter processor relatedto a predicted image in an image encoding device according to a fourthembodiment of the present disclosure, and FIG. 19B is an explanatorydiagram illustrating an example of the filter process of the predictedimage.

FIGS. 20A, 20B, and 20C are explanatory diagrams exemplifying anorthogonal transformation application block after block division withrespect to the predicted image according to the present disclosure.

FIG. 21 is a block diagram illustrating an example of the image encodingdevice according to the fourth embodiment of the present disclosure.

FIG. 22 is a flowchart of the filter process of the predicted image inthe image encoding device according to the fourth embodiment of thepresent disclosure.

FIG. 23 is a flowchart of an orthogonal transformation process in theimage encoding device according to the fourth embodiment of the presentdisclosure.

FIG. 24 is a block diagram of the periphery of the filter processorrelated to the predicted image in an image decoding device according tothe fourth embodiment of the present disclosure.

FIG. 25 is a block diagram illustrating an example of the image decodingdevice according to the fourth embodiment of the present disclosure.

FIGS. 26A and 26B are block diagrams of a periphery of a filterprocessor in an image encoding device and an image decoding deviceaccording to a fifth embodiment of the present disclosure.

FIG. 27 is a flowchart of a filter process of a predicted image in theimage encoding device or the image decoding device according to thefifth embodiment of the present disclosure.

FIG. 28 is an explanatory diagram of the filter process of the predictedimage in the image encoding device or the image decoding deviceaccording to the fifth embodiment of the present disclosure.

FIGS. 29A, 29B, 29C, and 29D are explanatory diagrams of an intraprediction in a conventional technique.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, an image encoding device and an image decoding deviceaccording to each embodiment of the present disclosure will be describedin this order.

First Embodiment (Image Encoding Device)

First, a filter processor 12 related to a predicted image, which is amain component of the present disclosure in an image encoding device 1according to a first embodiment of the present disclosure, will bedescribed with reference to FIGS. 1 and 2, and a specific typicalexample of the image encoding device 1 will be described with referenceto FIGS. 3 to 5.

FIG. 1A is a block diagram of a periphery of the filter processor 12related to the predicted image in the image encoding device 1 accordingto the first embodiment of the present disclosure, and FIG. 1B is anexplanatory diagram illustrating an example of a filter process of thepredicted image by the filter processor 12.

As illustrated in FIG. 1A, the image encoding device 1 according to thefirst embodiment of the present disclosure includes a neighboring pixelnon-reference predictor 11, the filter processor 12, a predictionresidual signal generator 13, and an orthogonal transformer 14.

The neighboring pixel non-reference predictor 11 is a functional partthat performs a signal prediction process of generating the predictedimage without using neighboring decoded signals (decoded neighboringsignals) in an encoding target region of an original image. In thepresent specification, such a signal prediction is referred to as a“neighboring pixel non-reference prediction”. The neighboring pixelnon-reference prediction includes, for example, an inter-prediction forperforming a motion compensation prediction between frames, an intrablock copy prediction for copying different decoded partial images ofthe same frame to generate a predicted image, and a signal prediction(referred to as a “cross component signal prediction” in the presentspecification) for generating a predicted image by using a correlationbetween component signals of luminance signals and color differencesignals at corresponding block positions of a certain frame.

For example, in the inter-prediction, as illustrated in FIG. 2A, when aframe F1 is an I frame (intra frame) in a plurality of frames F1 to F6of a certain GOP structure, in the case of generating a predicted imageby referring to a past I frame or a P frame like a P frame (predictiveinter frame), or a predicted image is generated with referring to aplurality of frames such as past and future frames such as a B frame(bi-predictive inter frame).

In addition, in the intra block copy prediction, as illustrated in FIG.2B, a different decoded partial image Bkr of the same frame F isreferred to and copied to generate a predicted image Bkp.

In addition, in the cross component signal prediction, as illustrated inFIG. 2C, by using correlation between component signals, a locallydecoded image Bkr of a luminance signal (for example, Y signal) at acorresponding block position of a certain frame F is referred to, and asynthesis process is performed on the predicted image of the colordifference signal (for example, U/V signal) by weighted addition togenerate a corrected predicted image Bkp of a color difference signal(for example, U/V signal). See Japanese Patent Application Laid-open No.2014-158270 for details related to such a cross component signalprediction.

These neighboring pixel non-reference predictions are common in that thesignal prediction is performed without using height of correlationbetween neighboring pixels while having a decoded region as a regionneighboring to a block of the encoding target region of the originalimage.

The filter processor 12 in the first embodiment is a functional partthat generates a predicted image including a new prediction signal byperforming a low-pass filter process on the prediction signal by usingthe decoded signals (decoded neighboring signals) neighboring to a leftside and an upper side of the predicted image among the pixel signals(prediction signals) in the block of the predicted image generated bythe neighboring pixel non-reference predictor 11, and outputs thepredicted image to the prediction residual signal generator 13.

For example, as illustrated in FIG. 1B, when a block Bkp of thepredicted image including 4×4 pixel signals p corresponding to a blockBko of the encoding target region of the original image is generated bythe neighboring pixel non-reference predictor 11, the filter processor12 generates the predicted image including the new prediction signal byperforming the low-pass filter process such as a smoothing filter in ahorizontal direction and a vertical direction, for example, by settingthe pixel signals (prediction signals) on a leftmost side and unuppermost side in the predicted image as a signal Sf of a filter targetregion, by using a decoded neighboring signal So in a periphery of ablock boundary of the block Bkp of the predicted image neighboring onthe left side and the upper side of the predicted image.

The prediction residual signal generator 13 calculates an error of eachpixel signal (prediction signal) of the block Bkp of the predicted imageobtained from the filter processor 12 for each pixel signal (originalsignal) of the block (encoding target block) Bko of the encoding targetregion of the original image, and outputs the error to the orthogonaltransformer 14 as a prediction residual signal.

The orthogonal transformer 14 performs a predetermined orthogonaltransformation process on the prediction residual signal input from theprediction residual signal generator 13, and generates a transformationcoefficient signal. For example, as the predetermined orthogonaltransformation process, an orthogonal transformation process used in anencoding method such as DCT or DST, or an integer orthogonaltransformation process defined by H.264 or H.265 approximated to aninteger can be used, as long as the process conforms to the encodingmethod to be used.

The image encoding device 1 according to the first embodiment performs aquantization process and an entropy encoding process on thetransformation coefficient signal and outputs the resultant signal tothe outside. Then, in the entropy encoding process, it is possible totransmit a video by transforming the video into a code by arithmeticencoding and the like represented by CABAC, together with variousencoding parameters and the like. Note that the encoding parameter canbe transmitted by including parameters that can be selected and set,such as inter-prediction parameters or intra prediction parameters,parameters of a block size related to block division (block divisionparameter), and quantization parameters related to the quantizationprocess.

Next, an inter predictor 11 a based on the inter-prediction will bedescribed as a representative example of the neighboring pixelnon-reference predictor 11, and a configuration and an operation exampleof the image encoding device 1 that performs the filter process on theprediction signal of the block of the predicted image by the filterprocessor 12 will be described with reference to FIGS. 3 to 5.

FIG. 3 is a block diagram illustrating an example of the image encodingdevice 1 according to the first embodiment of the present disclosure.The image encoding device 1 illustrated in FIG. 3 includes apre-processor 10, the inter predictor 11 a, the filter processor 12, theprediction residual signal generator 13, the orthogonal transformer 14,a quantizer 15, an inverse quantizer 16, an inverse orthogonaltransformer 17, a decoded image generator 18, an in-loop filter 19, aframe memory 20, an intra predictor 21, a motion vector calculator 22,an entropy encoder 23, and a predicted image selector 24.

The pre-processor 10 divides the original image for each frame of theinput moving image data into encoding target blocks having apredetermined block size and outputs the encoding target blocks to theprediction residual signal generator 13 in a predetermined order.

The prediction residual signal generator 13 calculates an error of eachpixel signal of a block of the predicted image obtained from the filterprocessor 12 for each pixel signal of the encoding target block andoutputs the error to the orthogonal transformer 14 as an orthogonaltransformation as a prediction residual signal of a block unit.

The orthogonal transformer 14 performs a predetermined orthogonaltransformation process on the prediction residual signal input from theprediction residual signal generator 13, and generates a transformationcoefficient signal.

The quantizer 15 performs a predetermined quantization process on thetransformation coefficient signal obtained from the orthogonaltransformer 14 and outputs the resultant signal to the entropy encoder23 and the inverse quantizer 16.

The inverse quantizer 16 performs an inverse quantization process on thequantized transformation coefficient signal obtained from the quantizer15, and outputs the resultant signal to the inverse orthogonaltransformer 17.

The inverse orthogonal transformer 17 reconstructs the predictionresidual signal by performing an inverse orthogonal transformationprocess on the inversely quantized transformation coefficient signalobtained from the inverse quantizer 16 and outputs the reconstructedprediction residual signal to the decoded image generator 18.

The decoded image generator 18 adds the prediction residual signalreconstructed by the inverse orthogonal transformer 17 to the block ofthe predicted image predicted by the intra predictor 21 or the interpredictor 11 a and obtained from the filter processor 12, generates ablock of a locally decoded image, and outputs the block of the locallydecoded image to the in-loop filter 19.

With respect to the block of the locally decoded image obtained from thedecoded image generator 18, the in-loop filter 19 performs in-loopfilter processes with, for example, an adaptive loop filter (ALF), apixel adaptive offset (SAO: sample adaptive offset), a deblockingfilter, or the like, and outputs the resultant signal to the framememory 20. The filter parameters related to these filter processes areoutput to the entropy encoder 23 as one of encoding parameters to beused as additional information of the encoding process.

The frame memory 20 stores the block of the locally decoded imageobtained through the in-loop filter 19 and holds the block of thelocally decoded image as a reference image that is usable by the intrapredictor 21, the inter predictor 11 a, and the motion vector calculator22.

At the time of selecting the signal for performing the signal predictiononly with the signal within the frame to be predicted by the predictedimage selector 24, the intra predictor 21 performs a DC prediction, aPlaner prediction, or a directional prediction with respect to pixelsignals (original signals) in an encoding target block of an originalimage by using pixel signals of encoded and decoded blocks neighboringto a left side or an upper side of the encoding target block stored as areference image in the frame memory 20, generates a block of a predictedimage including pixel signals (prediction signals) predicted byextrapolation, and outputs the block of the predicted image to thefilter processor 12.

The intra prediction parameter for identifying the DC prediction, thePlaner prediction, or the directional prediction used in the intrapredictor 21 is output to the entropy encoder 23 as one of the encodingparameters used as the additional information of the encoding process.

At the time of selecting the signal for performing the inter-framemotion compensation prediction on the PB frame by the predicted imageselector 24, the inter predictor 11 a generates the block of thepredicted image by motion-compensating data stored as a reference imagein the frame memory 20 with the motion vector provided from the motionvector calculator 22 with respect to the pixel signal (original signal)in the encoding target block of the original image, and outputs theblock of the predicted image to the filter processor 12.

The motion vector calculator 22 searches for a position most similar tothe encoding target block of the original image by using a blockmatching technique or the like with respect to the data of the referenceimage stored in the frame memory 20, calculates a value indicating aspatial shift as a motion vector, and outputs the value to the interpredictor 11 a.

The inter-prediction parameter used for the inter-prediction includingthe motion vector calculated by the motion vector calculator 22 isoutput to the entropy encoder 23 as one of the encoding parameters to bethe additional information of the encoding process.

The entropy encoder 23 performs an entropy encoding process on theoutput signal from the quantizer 15 and various encoding parameters andoutputs a stream signal of the encoded moving image data.

An operation example of the filter processor 12 will be described withreference to FIGS. 4 and 5. As illustrated in FIG. 4, when the predictedimage from the intra predictor 21 or the inter predictor 11 a is input(step S1), the filter processor 12 identifies whether the predictedimage is a neighboring pixel non-reference prediction, that is, in thisexample, identifies whether the predicted image is the inter-predictionbased on the signal selection by the predicted image selector 24 (stepS2). The same applies to a case where the neighboring pixelnon-reference prediction is an intra block copy prediction or a crosscomponent signal prediction to be described above.

Subsequently, when the filter processor 12 determines that the predictedimage is not the neighboring pixel non-reference prediction, that is,the intra prediction in this example (step S2: No), the filter processor12 determines whether or not to perform the filter process by apredetermined processing method (step S5). When the filter process isnot performed on the predicted image of the intra prediction (step S5:No), the filter processor 12 outputs the prediction signal to theprediction residual signal generator 13 without performing the filterprocess thereon (step S6). On the other hand, when filter process isperformed on the predicted image of the intra prediction (step S5: Yes),the filter processor 12 proceeds to step S3. Since the filter process onthe predicted image of the intra prediction may be performed in the samemanner as the currently defined H.265 and is not directly related to thegist of the present disclosure, the description of the step S3 isdescribed as an example that performs the filter process on thepredicted image of the inter-prediction.

When the filter processor 12 determines that the predicted image is theneighboring pixel non-reference prediction, that is, theinter-prediction in this example (step S2: Yes), the filter processor 12extracts a pixel signal of a filter process target region in advancefrom the predicted image (prediction signal) (step S3). For example, asillustrated in FIG. 5, the pixel signals (prediction signals) on theleftmost side and the uppermost side of the predicted image among thepixel signals (prediction signals) in the 4×4 predicted image block aredefined as a filter process target regions Sf.

Subsequently, the filter processor 12 generates a predicted imageincluding a new prediction signal by performing a low-pass filterprocess on the pixel signal (prediction signal) of the selected filterprocess target region by using the decoded signal (the decodedneighboring signal) neighboring on the left side and the upper side ofthe predicted image, and outputs the generated predicted image to theprediction residual signal generator 13 (step S4). For example, asillustrated in FIG. 5, the filter processor 12 performs a smoothingfilter process with a predetermined weighting factor for each of ahorizontal filter region, a vertical filter region, and an angularfilter region in the filter process target region Sf among the pixelsignals (prediction signals) in the block Bkp of the 4×4 predictedimage, and outputs the resultant signal to the prediction residualsignal generator 13.

It should be noted that the filter process by the filter processor 12may be a low-pass filter process, filter processes other than thesmoothing filter may be applied, and the region of the decodedneighboring signal or the weighting factor used for the filter processare not necessarily limited to the example illustrated in FIG. 5. Forexample, in the case of a large-size encoded block, the filter processmay be applied to a plurality of rows or columns. In addition, thefilter type, the filter process target region, and the weighting factor,which are related to the filter process performed by the filterprocessor 12, may be configured to be transmitted as additionalinformation of the encoding process, but need not be transmitted if theyare determined in advance between transmission and reception (betweenencoding and decoding).

By performing the filter process on the predicted image by the filterprocessor 12 in this way, a signal error occurring between the pixelsignals (prediction signals) on the leftmost side and the uppermost sidein the predicted image and the neighboring decoded signal is reduced,regardless of the type of the orthogonal transformation process of theorthogonal transformer 14, and it is possible to improve the encodingefficiency of the video.

(Image Decoding Device)

Next, a filter processor 54 and its peripheral functional blocks, whichare the main components of the image decoding device 5 according to thefirst embodiment of the present disclosure, will be described withreference to FIG. 6, and a specific typical example of the imagedecoding device 5 will be described with reference to FIG. 7.

FIG. 6 is a block diagram of the periphery of a filter processor 54 withrespect to a predicted image in the image decoding device 5 according tothe first embodiment of the present disclosure. As illustrated in FIG.6, the image decoding device 5 according to the first embodiment of thepresent disclosure includes an inverse orthogonal transformer 52, aneighboring pixel non-reference predictor 53, a filter processor 54, anda decoded image generator 55.

The inverse orthogonal transformer 52 receives the stream signaltransmitted from the image encoding device 1 side, performs an inverseorthogonal transformation process on transformation coefficientsreconstructed through a entropy decoding process and an inversequantization process, and outputs the obtained prediction residualsignal to the decoded image generator 55.

The neighboring pixel non-reference predictor 53 is a functional partcorresponding to the neighboring pixel non-reference predictor 11 on theimage encoding device 1 side, and performs a signal prediction processfor generating a predicted image without using neighboring decodedsignals (decoded neighboring signals). The neighboring pixelnon-reference prediction includes, for example, the inter-prediction forperforming the motion compensation prediction between frames, the intrablock copy prediction for generating a predicted image by copyingdifferent decoded partial images of the same frame, and the crosscomponent signal prediction for generating a predicted image by using acorrelation between component signals of a luminance signal and a colordifference signal at the corresponding block position of a certainframe.

The filter processor 54 of the first embodiment is a functional partcorresponding to the filter processor 12 of the image encoding device 1side, and generates a predicted image including a new prediction signalby performing a low-pass filter process on the prediction signal byusing decoded signals (decoded neighboring signals) neighboring to theleft side and the upper side of the predicted image, among the pixelsignals (prediction signals) in the block of the predicted imagegenerated by the neighboring pixel non-reference predictor 53, andoutputs the predicted image to the decoded image generator 55.

The decoded image generator 55 is a functional part that adds theprediction residual signal reconstructed by the inverse orthogonaltransformer 52 to the block of the predicted image obtained from thefilter processor 54 to be described later to generate the block of thedecoded image including the decoded signal.

That is, the prediction residual signal generated through the filterprocess on the predicted image by the filter processor 12 in the imageencoding device 1 is transmitted in a state in which the video encodingefficiency is improved, and the image decoding device 5 can efficientlyreconstruct the video related to this transmission. Similarly to thecase of the filter processor 12, the filter processor 54 may use afilter other than the smoothing filter as long as the filter process isa low-pass filter process, and the region of the decoded neighboringsignal or the weighting factor used for the filter process may beappropriately determined. Then, when the filter type, the filter processtarget region, and the weighting factor, which are related to the filterprocess by the filter processor 12 from the image encoding device 1side, are determined in advance between transmission and reception(between encoding and decoding), or are transmitted as additionalinformation of the encoding process, the accuracy of the decoded signalcan be improved by performing the filter process similarly to the filterprocessor 54 of the image decoding device 5 according to this.

Next, the inter predictor 53 a based on the inter-prediction will bedescribed as a representative example of the neighboring pixelnon-reference predictor 53, and a configuration and an operation exampleof the image decoding device 5 that performs the filter process on theprediction signal based on the inter-prediction by the filter processor54 will be described with reference to FIG. 7.

FIG. 7 is a block diagram illustrating an example of the image decodingdevice 5 according to the first embodiment of the present disclosure.The image decoding device 5 illustrated in FIG. 7 includes an entropydecoder 50, an inverse quantizer 51, an inverse orthogonal transformer52, an inter predictor 53 a, a filter processor 54, a decoded imagegenerator 55, an in-loop filter 56, a frame memory 57, an intrapredictor 58, and a predicted image selector 59.

The entropy decoder 50 receives a stream signal transmitted from theimage encoding device 1 side, outputs, to each function block, variousencoding parameters obtained by performing an entropy decoding processcorresponding to the entropy encoding process of the image encodingdevice 1, and outputs quantized transformation coefficients to theinverse quantizer 51. As various encoding parameters, for example, thefilter parameter, the intra prediction parameter, and theinter-prediction parameter are output to the in-loop filter 56, theintra predictor 58, and the inter predictor 53 a, respectively.

The inverse quantizer 51 performs an inverse quantization processcorresponding to the quantization process of the image encoding device 1on the quantized transformation coefficients obtained from the entropydecoder 50, and outputs the transformation coefficient to the inverseorthogonal transformer 52.

The inverse orthogonal transformer 52 reconstructs the predictionresidual signal by performing an inverse orthogonal transformationprocess corresponding to the orthogonal transformation process of theimage encoding device 1 to the transformation coefficient obtained fromthe inverse quantizer 51, and outputs the reconstructed predictionresidual signal to the decoded image generator 55.

The decoded image generator 55 adds the prediction residual signalreconstructed by the inverse orthogonal transformer 52 to the block ofthe predicted image predicted by the intra predictor 58 or the interpredictor 53 a and generated by performing a low-pass filter by thefilter processor 54, generates the block of the decoded image, andoutputs the block of the decoded image to the in-loop filter 56.

The in-loop filter 56 performs a filter process corresponding to thefilter process of the in-loop filter on the image encoding device 1 sideand outputs the resultant signal to the frame memory 57.

The frame memory 57 stores the block of the decoded image obtainedthrough the in-loop filter 56 and holds the block of the decoded imageas a reference image that is usable by the intra predictor 58 and theinter predictor 53 a.

At the time of selecting the signal for performing the signal predictiononly with the signal within the frame to be predicted by the predictedimage selector 59, the intra predictor 58 generates a block of apredicted image by performing the prediction process corresponding tothe intra predictor 21 on the image encoding device 1 side by using theintra prediction parameter and pixel signals of decoded blocksneighboring to a left side or an upper side of the decoding target blockstored in the frame memory 57 as the reference image, and outputs theblock of the predicted image to the filter processor 54.

At the time of selecting the signal for performing the inter-framemotion compensation prediction on the PB frame by the predicted imageselector 59, the inter predictor 53 a generates the block of thepredicted image by motion-compensating data stored as a reference imagein the frame memory 20 by using the motion vector included from theinter-prediction parameter, and outputs the block of the predicted imageto the filter processor 54.

It is possible to construct the frame from the decoded data stored inthe frame memory 57 and to output the frame as the decoded image.

The filter processor 54 operates in the same manner as that illustratedand described with reference to FIGS. 4 and 5. That is, in the filterprocess in the filter processor 54 according to the present disclosure,when the predicted image is the neighboring pixel non-referenceprediction, that is, in this example, when it is the inter-prediction,the pixel signal (prediction signal) of the predetermined filter processtarget region is selected from the predicted image, a predicted imageincluding a new prediction signal is generated by performing a low-passfilter process on the pixel signal (prediction signal) of the selectedfilter process target region by using decoded signals (decodedneighboring signals) neighboring on the left side or the upper side ofthe predicted image, and the generated predicted image is output to thedecoded image generator 55. For example, as illustrated in FIG. 5, thefilter processor 54 performs a smoothing filter process with apredetermined weighting factor for each of the horizontal filter region,the vertical filter region, and the angular filter region in the filterprocess target region Sf among the pixel signals (prediction signals) inthe block Bkp of the predicted image by using the decoded neighboringsignal So around the block boundary of the block Bkp of the predictedimage, and outputs the resultant signal to the decoded image generator55.

According to the image encoding device 1 and the image decoding device 5of the first embodiment configured as described above, since a signalerror occurring between the prediction signal of the predicted image andthe neighboring decoded block signal is reduced and the encodingefficiency is improved, it is possible to realize an image encodingdevice and an image decoding device of a video encoding method with highencoding efficiency. That is, the residual component in the predictionresidual signal can be made smaller, and the encoding efficiency can beimproved by reducing the amount of information to be encoded andtransmitted.

Second Embodiment

Next, the filter processors 12 and 54 in the image encoding device 1 andthe image decoding device 5 according to the second embodiment of thepresent disclosure will be described with reference to FIG. 8. In thefilter processors 12 and 54 of the first embodiment, an example thatgenerates the predicted image including the new prediction signal byperforming the low-pass filter process on the prediction signal by usingthe decoded signals (decoded neighboring signals) neighboring to theleft side and the upper side of the predicted image, among the pixelsignals (prediction signals) in the block of the predicted imagegenerated by the neighboring pixel non-reference prediction has beendescribed.

On the other hand, the filter processors 12 and 54 of the presentembodiment are configured to perform a correlation determination fordetermining whether a low-pass filter process is performed on theprediction signal to be subjected to the low-pass filter process byusing the decoded neighboring signal referred to in the low-pass filterprocess among the pixel signals (prediction signals) in the blocks ofthe predicted image generated by the neighboring pixel non-referenceprediction, and determine whether to apply the low-pass filter processaccording to the result of the correlation determination. Likecomponents are denoted by the same reference numerals. Accordingly,since the filter processors 12 and 54 of the present embodiment can beapplied to the image encoding device 1 and the image decoding device 5illustrated in FIGS. 3 and 7, only the processing contents related tothe filter processors 12 and 54 of the present embodiment will bedescribed, and a further detailed description will be omitted.

FIG. 8 is a block diagram of the filter processors 12 and 54 in theimage encoding device 1 or the image decoding device 5 according to thepresent embodiment. In addition, FIG. 9 is a flowchart of the filterprocesses 12 and 54 of the predicted image in the image encoding device1 or the image decoding device 5 according to the present embodiment.

The filter processors 12 and 54 of the present embodiment include ahorizontal direction correlation determiner 101, a vertical directioncorrelation determiner 102, a filter process determiner 103, and afilter process executor 104. The operation of these functional blockswill be described with reference to the processing example illustratedin FIG. 9. It should be noted that the filter processors 12 and 54 ofthe present embodiment may be configured to process whether theneighboring pixel non-reference prediction is performed, for example,whether the inter-prediction or the intra prediction is performed, butan example in which the filter process is performed on the predictedimage of the neighboring pixel non-reference prediction according to thepresent disclosure will be described.

When a predicted image is input (step S11), the filter processors 12 and54 perform a correlation determination process for each filter processtarget region individually determined by the horizontal directioncorrelation determiner 101 and the vertical direction correlationdeterminer 102 (step S12).

That is, the horizontal direction correlation determiner 101 determinesthe correlation with the decoded neighboring signal on the left side ofthe predicted image (step S13). When the horizontal directioncorrelation determiner 101 determines that the correlation in thehorizontal direction is high (step S13: Yes), the execution of thehorizontal filter process is determined by using the decoded neighboringsignal (step S14). When the horizontal direction correlation determiner101 determines that the correlation in the horizontal direction is low(step S13: No), it is output to the filter process determiner 103 thatthe horizontal filter process is not to be executed.

Similarly, the vertical direction correlation determiner 102 determinesthe correlation with the decoded neighboring signal on the upper side ofthe predicted image (step S15). When the vertical direction correlationdeterminer 102 determines that the correlation in the vertical directionis high (step S15: Yes), the execution of the vertical filter process isdetermined by using the decoded neighboring signal (step S16). When thevertical direction correlation determiner 102 determines that thecorrelation in the vertical direction is low (step S15: No), it isoutput to the filter process determiner 103 that the vertical filterprocess is not to be executed.

The filter process determiner 103 determines whether it is determinedthat both the horizontal and vertical filter processes are to beexecuted (step S17). If the filter process determiner 103 receives adetermination to execute both the horizontal and vertical filterprocesses (step S17: Yes), the filter process determiner 103 determinesthat both the horizontal and vertical filter processes and a 3-tapangular filter process are executed (step S18). If the filter processdeterminer 103 receives a determination not to execute both thehorizontal and vertical filter processes (step S17: No), the filterprocess determiner 103 determines that the filter process is performedonly in the horizontal direction when the horizontal correlation is highand the filter process is performed only in the vertical direction whenthe correlation in the vertical direction is high. In the case of onlyone of the horizontal and vertical filter processes, the filter processdeterminer 103 determines that the angular filter performs thehorizontal or vertical filter process (step S19).

The process order of steps S13 to S19 illustrated in FIG. 9 is merely anexample and can be configured in random order. Therefore, the filterprocess determiner 103 performs correlation determination for eachfilter process target region individually determined in the horizontaldirection and the vertical direction, determines the filter process tobe “execute only in the horizontal direction”, “execute only in thevertical direction”, or “execute both in the horizontal direction, thevertical direction, and the corner”, or “not execute in the horizontaldirection, the vertical direction, and the corner”, and the filterprocess determiner 103 outputs the determination to the filter processexecutor 104.

The filter process executor 104 performs a filter process according tothe determination result of the filter process determiner 103 to outputa filtered predicted image (step S20), and when the determination resultof the filter process determiner 103 is “not execute in the horizontaldirection, the vertical direction, or the corner”, the filter processdeterminer 103 outputs the predicted image input to the filterprocessors 12 and 54 as it is.

For example, the horizontal direction correlation determiner 101 obtainsdifferences between neighboring pixels for each of four horizontaldirection determination regions A1 to A4 illustrated in FIG. 10A. If thesum of the differences in the horizontal direction determination regionsA1 to A4 is equal to or less than a predetermined threshold value, thehorizontal direction correlation determiner 101 determines that thecorrelation is high, and otherwise, the horizontal direction correlationdeterminer 101 determines that the correlation is low. Similarly, thevertical direction correlation determiner 102 obtains differencesbetween neighboring pixels for each of four vertical directiondetermination regions B1 to B4 illustrated in FIG. 10B. If the sum ofthe differences in the vertical direction determination regions B1 to B4is equal to or less than a predetermined threshold value, the verticaldirection correlation determiner 102 determines that the correlation ishigh, and otherwise, the vertical direction correlation determiner 102determines that the correlation is low. Then, by applying the filterprocess in the direction of high correlation as illustrated in FIG. 5,the filter process can be determined to “execute only in the horizontaldirection”, “execute only in the vertical direction”, “execute both inthe horizontal direction, the vertical direction, or the corner”, and“not execute in the horizontal direction, the vertical direction, andthe corner”. Then, when the filter process is performed only in thehorizontal or vertical direction, a horizontal or vertical filter isapplied to a pixel (prediction signal) located at the corner of theblock of the predicted image.

The correlation determination region need not be necessarily identicalto the filter process target region the same, but if so, the processbecomes simple. In addition, it should be noted that various techniquesare assumed for concrete processing methods of the correlationdetermination process and the filter process, and only one example isillustrated.

By configuring the filter processors 12 and 54 as in the presentembodiment and applying the filter processors 12 and 54 to the imageencoding device 1 and the image decoding device 5, respectively, it ispossible to improve the encoding efficiency and suppress thedeterioration of the image quality caused by this.

Third Embodiment (Image Encoding Device)

Next, the filter processor 12 and an orthogonal transformation selectioncontroller 25 related to a predicted image, which are main components ofthe present disclosure in the image encoding device 1 according to athird embodiment of the present disclosure, will be described withreference to FIGS. 1B and 11, and image encoding devices 1 of twospecific examples will be described with reference to FIGS. 5 and 12 to15. The following description focuses on differences from theabove-described embodiment. That is, in the present embodiment, the samereference numerals are assigned to the same components as those in theabove embodiment, and a further detailed description thereof will beomitted.

FIG. 11A is a block diagram of the periphery of the filter processor 12and an orthogonal transformation selection controller 25 with respect toa predicted image in the image encoding device 1 according to thepresent embodiment, and FIG. 1B is an explanatory diagram illustratingan example of the filter process of the predicted image in the filterprocessor 12.

As illustrated in FIG. 11A, the image encoding device 1 according to thepresent embodiment includes the neighboring pixel non-referencepredictor 11, the filter processor 12, the prediction residual signalgenerator 13, the orthogonal transformer 14, and an orthogonaltransformation selection controller 25.

The filter processor 12 in the present embodiment is a functional partthat, under the control of the orthogonal transformation selectioncontroller 25, generates a predicted image including a new predictionsignal by performing a low-pass filter process on the prediction signalby using decoded signals (decoded neighboring signals) neighboring tothe left side and the upper side of the predicted image, among the pixelsignals (prediction signals) in the block of the predicted imagegenerated by the neighboring pixel non-reference predictor 11, andoutputs the predicted image to the prediction residual signal generator13.

Under the control of the orthogonal transformation selection controller25, the orthogonal transformer 14 performs one of two types oforthogonal transformation processes on the prediction residual signalinput from the prediction residual signal generator 13, and generates atransformation coefficient signal.

For example, as the two types of the orthogonal transformationprocesses, there are an orthogonal transformation process in which theend of the transformation base is closed (for example, DST (DiscreteSine Transform)), and an orthogonal transformation process in which theend of the transformation base is opened (for example, DCT (DiscreteCosine Transform)), and it is not limited as to whether it is realnumber precision or integer precision. For example, it can be an integerorthogonal transformation process defined by H.264 or H.265 whichapproximates an integer, as long as it conforms to the encoding methodto be used.

Here, an example of a base waveform of an orthogonal transformation isillustrated in FIGS. 11B and 11C. In the examples illustrated in FIGS.11B and 11C, with respect to four transformation bases (N=4) usable fororthogonal transformation of a block of a 4×4 prediction residualsignal, transformation base waveforms from low to high frequencies (u=0to 3 for DCT, u=1 to 4 for DST) are illustrated as patterns of frequencycomponents of cosine and sine in the DCT transformation base (FIG. 11B)and the DST transformation base (FIG. 11C). As illustrated in FIGS. 11Band 11C, a main difference other than u=0 for DCT and u=4 for DST is aphase, and it can be seen that each transformation base (for exampleu=2) of the corresponding frequency has the same correlation betweenpixels at the same frequency (base amplitude is the same), and the phaseis shifted by π/2. Then, even in any transformation base waveform, inthe DCT, as illustrated in FIG. 11B, an orthogonal transformationprocess is realized in which the end point is opened with a large valueand the end of the transformation base is opened. On the other hand, inthe DST, as illustrated in FIG. 11C, an orthogonal transformationprocess is realized in which the end point is closed with a small valueand the end of the transformation base is closed.

It should be noted that “the orthogonal transformation process in whichthe end of the transformation base is closed” refers to a state in whichone end on the side close to the prediction reference block is closed inthe block of the prediction residual signal, for example, may be anasymmetric DST type 7 of the transformation base with the other endopened as illustrated in FIG. 11C.

At the time of the neighboring pixel non-reference prediction, under apredetermined evaluation criterion, when the filter processor 12performs a low-pass filter process on a pixel value (prediction signal)located at a block boundary of the predicted image, the orthogonaltransformation selection controller 25 selectively controls theorthogonal transformation type to instruct the orthogonal transformer 14to use an orthogonal transformation process (for example, DST), in whichthe end of the transformation base is closed, with respect to the blockof the prediction residual signal obtained from the prediction residualsignal generator 13 based on the block of the predicted image after thelow-pass filter process. Under the predetermined evaluation criterion,when the low-pass filter process is not executed on the block of thepredicted image, the orthogonal transformation selection controller 25selectively controls the orthogonal transformation type to instruct theorthogonal transformer 14 to use an orthogonal transformation process(e.g., DCT), in which the end of the transformation base is opened, withrespect to the block of the prediction residual signal. The orthogonaltransformation selection controller 25 outputs, to the decoding side, atransformation type identification signal indicating which one of thetwo types of the orthogonal transformation processes has been applied asone of the encoding parameters.

Two examples will be described as the predetermined evaluation criterionin the orthogonal transformation selection controller 25.

As will be described in detail later, in the first embodiment, theorthogonal transformation selection controller 25 causes the filterprocessor 12 to select a signal of a filter process target region asillustrated in FIG. 1B, and instruct provisional execution of thelow-pass filter process by using a decoded neighboring signal, andacquires each predicted image of execution/non-execution of the filterprocess as prediction block information (est1) from the filter processor12. Subsequently, the orthogonal transformation selection controller 25compares each predicted image of execution/non-execution of the filterprocess from the prediction block information (est1), and when theorthogonal transformation selection controller 25 determines that theinfluence of the end point of the prediction region (that is, aprediction residual signal for a pixel signal located at the outerperiphery of the block of the predicted image) is relatively large withrespect to the block size (4×4 is exemplified in this example, but theblock size is not limited), the predicted image after the execution ofthe filter process is output from the filter processor 12 to theprediction residual signal generator 13. Then, the orthogonaltransformation selection controller 25 instructs the orthogonaltransformer 14 to use the orthogonal transformation process (forexample, DST), in which the end of the transformation base is closed,with respect to the block of the prediction residual signal obtainedfrom the prediction residual signal generator 13 based on the block ofthe predicted image after the low-pass filter process.

Subsequently, the orthogonal transformation selection controller 25compares each predicted image of execution/non-execution of the filterprocess obtained from the filter processor 12, and when the orthogonaltransformation selection controller 25 determines that the influence ofthe end point of the prediction region (that is, a prediction residualsignal for a pixel signal located at the outer periphery of the block ofthe predicted image) is relatively small with respect to the block size(4×4 is exemplified in this example, but the block size is not limited),the predicted image of the non-execution of the filter process is outputfrom the filter processor 12 to the prediction residual signal generator13. Then, the orthogonal transformation selection controller 25instructs the orthogonal transformer 14 to use the orthogonaltransformation process (for example, DCT), in which the end of thetransformation base is opened, with respect to the block of theprediction residual signal obtained from the prediction residual signalgenerator 13 based on the block of the predicted image of thenon-execution of the low-pass filter process.

Here, various evaluation methods are assumed as to whether the influenceof the end point of the prediction region is relatively large withrespect to the block size (4×4 is exemplified in this example, but theblock size is not limited), but for example, it may be determinedwhether it is equal to or higher than a predetermined level by a ratiocomparison between a variance value of the density distribution of theend point of the prediction region and a variance value of the densitydistribution of the entire block size. As described above, in the firstembodiment, the orthogonal transformation selection controller 25determines the presence or absence of execution of the filter processbased on the criterion for determining whether the filter process effectby the filter process is equal to or higher than a predetermined level,further determines the type of the orthogonal transformation process tobe applied according to the presence or absence of execution of thefilter process, and controls the filter processor 12 and the orthogonaltransformer 14.

In addition, in the second embodiment, the orthogonal transformationselection controller 25 causes the filter processor 12 to select thesignal of the filter process target region as illustrated in FIG. 1B,instructs provisional execution of the low-pass filter process by usingthe decoded neighboring signal, acquire and compares rate distortioninformation (est2) of the generated encoding amount (R) and the encodingdistortion amount (D) in a case where the quantization process and theentropy encoding process are performed by orthogonal transformationusing the orthogonal transformation process (for example, DST) in whichthe end of the transformation base is closed when the filter process isexecuted, and in a case where the quantization process and the entropyencoding process are performed by orthogonal transformation using theorthogonal transformation process (for example, DCT) in which the end ofthe transformation base is opened when the filter process is notexecuted, and selects a combination that is excellent as an RD cost (forexample, a combination of execution of the filter process and DST, and acombination of non-execution of the filter process and the DCT).

With such RD optimization, the orthogonal transformation selectioncontroller 25 causes the filter processor 12 to output the predictedimage after the execution of the filter process to the predictionresidual signal generator 13 when the combination of “the execution ofthe filter process and the orthogonal transformation process in whichthe end of transformation base is closed” is excellent, and instructsthe orthogonal transformer 14 to use the orthogonal transformationprocess (for example, DST) in which the end of the transformation baseis closed with respect to the block of the prediction residual signalobtained from the prediction residual signal generator 13 based on theblock of the predicted image after the low-pass filter process.

On the other hand, the orthogonal transformation selection controller 25causes the filter processor 12 to output the predicted image of thenon-execution of the filter process to the prediction residual signalgenerator 13 when the combination of “the non-execution of the filterprocess and the orthogonal transformation process in which the end ofthe transformation base is opened” is excellent, and instructs theorthogonal transformer 14 to use the orthogonal transformation process(for example, DCT) in which the end of the transformation base is openedwith respect to the block of the prediction residual signal obtainedfrom the prediction residual signal generator 13 based on the block ofthe predicted image of the non-execution of the low-pass filter process.

As described above, in the second embodiment, the orthogonaltransformation selection controller 25 determines the presence orabsence of execution of the filter process based on the RD optimization,further determines the type of the orthogonal transformation process tobe applied according to the presence or absence of execution of thefilter process, and controls the filter processor 12 and the orthogonaltransformer 14.

Finally, the image encoding device 1 according to the present embodimentperforms a quantization process and an entropy encoding process on atransformation coefficient signal obtained from the orthogonaltransformer 14 and outputs the resultant signal to the outside. Then, inthe entropy encoding process, it is possible to transmit a video bytransforming into a code by arithmetic encoding represented by CABACtogether with various encoding parameters and the like. In addition to atransformation type identification signal according to the presentdisclosure, parameters that can be selected and set, such asinter-prediction parameters, intra prediction parameters, parameters ofa block size related to block division (block division parameter),quantization parameters related to a quantization process, and the like,can be included in the encoding parameters and transmitted.

Hereinafter, more specifically, the configuration of the image encodingdevice 1, in which the inter predictor 11 a based on theinter-prediction is adopted as the representative example of theneighboring pixel non-reference predictor 11 and the examples describedabove are applied as the predetermined evaluation criterion in theorthogonal transformation selection controller 25, will be described.

First Example

FIG. 12 is a block diagram illustrating a first example of the imageencoding device 1 according to the present embodiment. The imageencoding device 1 illustrated in FIG. 12 includes the pre-processor 10,the inter predictor 11 a, the filter processor 12, the predictionresidual signal generator 13, the orthogonal transformer 14, thequantizer 15, the inverse quantizer 16, the inverse orthogonaltransformer 17, the decoded image generator 18, the in-loop filter 19,the frame memory 20, the intra predictor 21, the motion vectorcalculator 22, the entropy encoder 23, the predicted image selector 24,an orthogonal transformation selection controller 25.

Under the control of the orthogonal transformation selection controller25, the orthogonal transformer 14 performs a predetermined orthogonaltransformation process on the prediction residual signal input from theprediction residual signal generator 13, and generates a transformationcoefficient signal.

Under the control of the orthogonal transformation selection controller25, the filter processor 12 in the first example controls the executionand non-execution of the low-pass filter process, and generates apredicted image including a new prediction signal by performing alow-pass filter process on the prediction signal by using decodedsignals (decoded neighboring signals) neighboring to the left side andthe upper side of the predicted image, among the pixel signals(prediction signals) in the block of the predicted image generated bythe inter predictor 11 a that is the neighboring pixel non-referenceprediction at the time of the execution, and outputs the predicted imageto the prediction residual signal generator 13. The predicted imagegenerated by the inter predictor 11 a at the time of the non-executionis output to the prediction residual signal generator 13. At the time ofthe prediction other than the neighboring pixel non-reference predictionsuch as the intra prediction, the filter processor 12 determines thepresence or absence of the execution of the filter process according toa predetermined method.

The operation of the orthogonal transformation selection controller 25in the first example will be described with reference to FIG. 13. FIG.13 is a flowchart of the process of the orthogonal transformationselection controller 25 of the first example in the image encodingdevice 1 of the present embodiment. First, the orthogonal transformationselection controller 25 receives, for example, notification from thefilter processor 12 or the predicted image selector 24, and determineswhether the predicted image processed by the filter processor 12 is theneighboring pixel non-reference prediction (step S1).

When the predicted image is the neighboring pixel non-referenceprediction (step S1: Yes), the orthogonal transformation selectioncontroller 25 causes the filter processor 12 to select a signal of afilter process target region and instructs the provisional execution ofthe low-pass filter process by using the decoded neighboring signal(step S2).

For example, as illustrated in FIG. 5, the filter processor 12 defines,as a filter process target regions Sf, the pixel signals (predictionsignals) on the leftmost side and the uppermost side of the predictedimage among the pixel signals (prediction signals) in the 4×4 predictedimage block. Subsequently, the filter processor 12 generates a predictedimage including a new prediction signal by performing a low-pass filterprocess on the pixel signal (prediction signal) of the selected filterprocess target region by using the decoded signal (the decodedneighboring signal) neighboring on the left side and the upper side ofthe predicted image. For example, as illustrated in FIG. 5, the filterprocessor 12 generates a filtered predicted image by performing asmoothing filter process with a predetermined weighting factor for eachof a horizontal filter region, a vertical filter region, and an angularfilter region in the filter process target region Sf among the pixelsignals (prediction signals) in the block Bkp of the 4×4 predictedimage.

It should be noted that the filter process by the filter processor 12may be a low-pass filter process, filter processes other than thesmoothing filter may be applied, and the region of the decodedneighboring signal or the weighting factor used for the filter processare not necessarily limited to the example illustrated in FIG. 5. Forexample, in the case of a large-size encoded block, the filter processmay be applied to a plurality of rows or columns. In addition, thefilter type, the filter process target region, and the weighting factor,which are related to the filter process performed by the filterprocessor 12, may be configured to be transmitted as additionalinformation of the encoding process, but need not be transmitted if theyare determined in advance between transmission and reception (betweenencoding and decoding).

Subsequently, referring to FIG. 13, the orthogonal transformationselection controller 25 acquires, from the filter processor 12, eachpredicted image of execution/non-execution of the filter process asprediction block information (est1) (step S3).

Subsequently, the orthogonal transformation selection controller 25compares each predicted image of execution/non-execution of the filterprocess from the prediction block information (est1), and determineswhether the effect of the filter process is equal to or higher than apredetermined level according to whether the influence of the end pointof the prediction region (that is, a prediction residual signal for apixel signal located at the outer periphery of the block of thepredicted image) is relatively large with respect to the block size (4×4is illustrated in this example, but the block size is not limited) (stepS4).

When it is determined that the influence of the end point of theprediction region is relatively large with respect to the block size(step S4: Yes), the orthogonal transformation selection controller 25causes the filter processor 12 to output the predicted image after theexecution of the filter process to the prediction residual signalgenerator 13, and instructs the orthogonal transformer 14 to use theorthogonal transformation process (for example, DST) in which the end ofthe transformation base is closed, with respect to the block of theprediction residual signal obtained from the prediction residual signalgenerator 13 based on the block of the predicted image after thelow-pass filter process (step S5).

On the other hand, when it is determined that the influence of the endpoint of the prediction region is relatively smaller with respect to theblock size (step S4: No), the orthogonal transformation selectioncontroller 25 causes the filter processor 12 to output the predictedimage of the non-execution of the filter process to the predictionresidual signal generator 13, and instructs the orthogonal transformer14 to use the orthogonal transformation process (for example, DCT) inwhich the end of the transformation base is opened, with respect to theblock of the prediction residual signal obtained from the predictionresidual signal generator 13 based on the block of the predicted imageof the non-execution of the low-pass filter process (step S6).

At the time of the prediction other than the neighboring pixelnon-reference prediction such as the intra prediction (step S1: No), theorthogonal transformation selection controller 25 controls the filterprocessor 12 and the orthogonal transformer 14 so that the presence orabsence of the execution of the filter process and the orthogonaltransformation type are executed according to a predetermined method(step S7).

As described above, in the first example, the orthogonal transformationselection controller 25 controls the filter processor 12 and theorthogonal transformer 14 to determine the presence or absence ofexecution of the filter process and the type of the orthogonaltransformation process to be applied, based on the criterion fordetermining whether the filter process effect by the filter process isequal to or higher than a predetermined level. Then, the orthogonaltransformation selection controller 25 outputs, to the decoding sidethrough the entropy encoder 23, a transformation type identificationsignal indicating which one of the two types of the orthogonaltransformation processes has been applied as one of the encodingparameters.

As described above, since the filter processor 12 and the orthogonaltransformer 14 are controlled to determine the presence or absence ofexecution of the filter process and the type of the orthogonaltransformation process to be applied, based on the criterion fordetermining whether the filter process effect by the filter process isequal to or higher than a predetermined level, the residual component inthe prediction residual signal can be further reduced, and the encodingefficiency can be improved by reducing the amount of information to beencoded and transmitted.

In particular, even at the time of the neighboring pixel non-referenceprediction, since the correlation between neighboring pixel signals isused for the block of the prediction residual signal obtained based onthe block of the predicted image after the filter process and theorthogonal transformation process (for example, DST) in which the end ofthe transformation base is closed is used, not only the phase but alsothe feature of the transformation base corresponding to the end point ofthe prediction region are matched, thereby improving the encodingefficiency more remarkably,

Second Example

Next, a second example will be described. FIG. 14 is a block diagramillustrating a second example of the image encoding device 1 accordingto the present embodiment. In FIG. 14, the same reference numerals areassigned to the same components as those in FIG. 12, and a furtherdetailed description thereof will be omitted.

Similarly to the first example, under the control of the orthogonaltransformation selection controller 25, the filter processor 12 in thesecond example controls the execution and non-execution of the low-passfilter process, and generates a predicted image including a newprediction signal by performing a low-pass filter process on theprediction signal by using decoded signals (decoded neighboring signals)neighboring to the left side and the upper side of the predicted image,among the pixel signals (prediction signals) in the block of thepredicted image generated by the inter predictor 11 a that is theneighboring pixel non-reference prediction, and outputs the predictedimage to the prediction residual signal generator 13. The predictedimage generated by the inter predictor 11 a at the time of thenon-execution is output to the prediction residual signal generator 13.At the time of the prediction other than the neighboring pixelnon-reference prediction such as the intra prediction, the filterprocessor 12 determines the presence or absence of the execution of thefilter process according to a predetermined method.

The operation of the orthogonal transformation selection controller 25in the second example will be described with reference to FIG. 15. FIG.15 is a flowchart of the process of the orthogonal transformationselection controller 25 of the second example in the image encodingdevice 1 of the present embodiment. First, the orthogonal transformationselection controller 25 receives, for example, notification from thefilter processor 12 or the predicted image selector 24, and determineswhether the predicted image processed by the filter processor 12 is theneighboring pixel non-reference prediction (step S11).

When the predicted image is the neighboring pixel non-referenceprediction (step S11: Yes), the orthogonal transformation selectioncontroller 25 causes the filter processor 12 to select a signal of afilter process target region as illustrated in FIG. 1B, and instructsthe provisional execution of the low-pass filter process by using thedecoded neighboring signal (step S12).

Subsequently, the orthogonal transformation selection controller 25causes the filter processor 12 to output the predicted image after theexecution of the filter process to the prediction residual signalgenerator 13 by using the orthogonal transformation process (forexample, DST), in which the end of the transformation base is closed,with respect to the orthogonal transformer 14 upon the execution of thefilter process of the filter processor 12, performs the orthogonaltransformation on the block of the prediction residual signal obtainedfrom the prediction residual signal generator 13 based on the block ofthe predicted image after the low-pass filter process, outputs thepredicted image of the non-execution of the filter process from thefilter processor 12 to the prediction residual signal generator 13 byusing the orthogonal transformation process (for example, DCT) in whichthe end of the transformation base is opened with respect to theorthogonal transformer 14 in a case where the quantization process andthe entropy encoding process are performed by the quantizer 15 and theentropy encoder 23 and at the time of the non-execution of the filterprocess of the filter processor 12, performs the orthogonaltransformation on the block of the prediction residual signal obtainedfrom the prediction residual signal generator 13 based on the block ofthe predicted image of the non-execution of the low-pass filter process,and acquires rate distortion information (est2) of each generatedencoding amount (R) and the encoding distortion amount (D) from thequantizer 15 and the entropy encoder 23 in a case where the quantizationprocess and the entropy encoding process are performed by the quantizer15 and the entropy encoder 23 (step S13).

Subsequently, the orthogonal transformation selection controller 25compares and determines, from the rate distortion information (est2),which combination (for example, a combination of the execution of thefilter process and the DST, a combination of the non-execution of thefilter process and the DCT) is excellent as RD cost (step S14).

When the orthogonal transformation selection controller 25 determinesthat the execution of the filter process and the use of the orthogonaltransformation process (for example, DST), in which the end of thetransformation base is closed, is excellent as the RD cost (step S14:Yes), the predicted image after the execution of the filter process isoutput from the filter processor 12 to the prediction residual signalgenerator 13, and the orthogonal transformer 14 is instructed to use theorthogonal transformation process (for example, DST), in which the endof the transformation base is closed, with respect to the block of theprediction residual signal obtained from the prediction residual signalgenerator 13 based on the block of the predicted image after thelow-pass filter process (step S15).

On the other hand, when the orthogonal transformation selectioncontroller 25 determines that the non-execution of the filter processand the use of the orthogonal transformation process (for example, DCT),in which the end of the transformation base is opened, is excellent asthe RD cost (step S14: No), the predicted image of the non-execution ofthe filter process is output from the filter processor 12 to theprediction residual signal generator 13, and the orthogonal transformer14 is instructed to use the orthogonal transformation process (forexample, DCT), in which the end of the transformation base is opened,with respect to the block of the prediction residual signal obtainedfrom the prediction residual signal generator 13 based on the block ofthe predicted image of the non-execution of the low-pass filter process(step S16).

At the time of the prediction other than the neighboring pixelnon-reference prediction such as the intra prediction (step S11: No),the orthogonal transformation selection controller 25 controls thefilter processor 12 and the orthogonal transformer 14 so that thepresence or absence of the execution of the filter process and theorthogonal transformation type are executed according to a predeterminedmethod (step S17).

As described above, in the second example, the orthogonal transformationselection controller 25 controls the filter processor 12 and theorthogonal transformer 14 to determine the presence or absence ofexecution of the filter process and the type of the orthogonaltransformation process to be applied, based on the result selected bythe RD optimization. Then, the orthogonal transformation selectioncontroller 25 outputs, to the decoding side through the entropy encoder23, a transformation type identification signal indicating which one ofthe two types of the orthogonal transformation processes has beenapplied as one of the encoding parameters.

As described above, since the orthogonal transformation selectioncontroller 25 controls the filter processor 12 and the orthogonaltransformer 14 to determine the presence or absence of execution of thefilter process and the type of the orthogonal transformation process tobe applied, based on the result selected by the RD optimization, theresidual component in the prediction residual signal can be furtherreduced, and the encoding efficiency can be improved by reducing theamount of information to be encoded and transmitted.

In particular, even at the time of the neighboring pixel non-referenceprediction, since the execution of the filter process is permitted, thecorrelation between neighboring pixel signals is used for the block ofthe prediction residual signal obtained based on the block of thepredicted image after the filter process and the orthogonaltransformation process (for example, DST) in which the end of thetransformation base is closed is used, not only the phase but also thefeature of the transformation base corresponding to the end point of theprediction region are matched, thereby improving the encoding efficiencymore remarkably.

(Image Decoding Device)

Next, the peripheral functional blocks of the filter processor 54 andthe inverse orthogonal transformation selection controller 60, which arethe main components of the image decoding device 5 according to thepresent embodiment, will be described with reference to FIG. 16, and aspecific typical example of the image decoding device 5 will bedescribed with reference to FIG. 17.

FIG. 16 is a block diagram of the periphery of the filter processor 54and the inverse orthogonal transformation selection controller 60 withrespect to the predicted image in the image decoding device 5 accordingto the present embodiment. As illustrated in FIG. 16, the image decodingdevice 5 according to the present embodiment includes an inverseorthogonal transformer 52, a neighboring pixel non-reference predictor53, a filter processor 54, a decoded image generator 55, and an inverseorthogonal transformation selection controller 60.

Under the control of the inverse orthogonal transformation selectioncontroller 60, the inverse orthogonal transformer 52 receives the streamsignal transmitted from the image encoding device 1 side, performs aninverse orthogonal transformation process on transformation coefficientsreconstructed through a entropy decoding process and an inversequantization process, and outputs the obtained prediction residualsignal to the decoded image generator 55.

The filter processor 54 is a functional part corresponding to the filterprocessor 12 on the image encoding device 1 side. Under the control ofthe inverse orthogonal transformation selection controller 60, thefilter processor 54 controls the execution and non-execution of thelow-pass filter process, generates a predicted image including a newprediction signal by performing a low-pass filter process on theprediction signal by using decoded signals (decoded neighboring signals)neighboring to the left side and the upper side of the predicted image,among the pixel signals (prediction signals) in the block of thepredicted image generated by the neighboring pixel non-referencepredictor 53 at the time of the execution, and outputs the predictedimage to the decoded image generator 55. The predicted image generatedby the neighboring pixel non-reference predictor 53 at the time of thenon-execution is output to the decoded image generator 55. At the timeof the prediction other than the neighboring pixel non-referenceprediction such as intra prediction, the filter processor 54 correspondsto the encoding side and determines the presence or absence of theexecution of the filter process according to a predetermined method.

At the time of the neighboring pixel non-reference prediction, when theuse of the orthogonal transformation process (for example, DST), inwhich the end of the transformation base is closed, is indicated withrespect to the transformation coefficient to be processed, withreference to the transformation type identification signal obtained fromthe image encoding device 1 side, the inverse orthogonal transformationselection controller 60 performs selective control by instructing theinverse orthogonal transformer 52 to apply the inverse orthogonaltransformation process (for example, IDST), in which the end of thetransformation base is closed with respect to the reconstructedtransformation coefficient, and instructs the filter processor 54 toexecute the corresponding low-pass filter process.

On the other hand, when the use of the orthogonal transformation process(for example, DCT), in which the end of the transformation base isopened, is indicated with reference to the transformation typeidentification signal obtained from the image encoding device 1 side,the inverse orthogonal transformation selection controller 60 performsselective control by instructing the inverse orthogonal transformer 52to apply the inverse orthogonal transformation process (for example,IDCT), in which the end of the transformation base is opened withrespect to the reconstructed transformation coefficient, and instructsthe filter processor 54 not to execute the corresponding low-pass filterprocess.

That is, the prediction residual signal generated through the filterprocess on the predicted image by the filter processor 12 in the imageencoding device 1 is transmitted through the orthogonal transformationprocess (for example, DST), in which the end of the transformation baseis closed, in a state in which the video encoding efficiency isimproved, and the image decoding device 5 can efficiently reconstructthe video related to this transmission. Similarly to the case of thefilter processor 12, the filter processor 54 may use a filter other thanthe smoothing filter as long as the filter process is a low-pass filterprocess, and the region of the decoded neighboring signal or theweighting factor used for the filter process may be appropriatelydetermined. Then, when the filter type, the filter process targetregion, and the weighting factor, which are related to the filterprocess by the filter processor 12 from the image encoding device 1side, are determined in advance between transmission and reception(between encoding and decoding), or are transmitted as additionalinformation of the encoding process, the accuracy of the decoded signalcan be improved by performing the filter process similarly to the filterprocessor 54 of the image decoding device 5 according to this.

Next, the inter predictor 53 a based on the inter-prediction will bedescribed as a representative example of the neighboring pixelnon-reference predictor 53, and a configuration example of the imagedecoding device 5 to which the inverse orthogonal transformationselection controller 60 that selectively controls the inverse orthogonaltransformer 52 and the filter processor 54 is applied with respect tothe prediction signal by the inter-prediction will be described.

FIG. 17 is a block diagram illustrating an example of the image decodingdevice 5 according to the present embodiment. The image decoding device5 illustrated in FIG. 17 includes an entropy decoder 50, an inversequantizer 51, an inverse orthogonal transformer 52, an inter predictor53 a, a filter processor 54, a decoded image generator 55, an in-loopfilter 56, a frame memory 57, an intra predictor 58, a predicted imageselector 59, and an inverse orthogonal transformation selectioncontroller 60.

Under the control of the inverse orthogonal transformation selectioncontroller 60, the inverse orthogonal transformer 52 reconstructs theprediction residual signal by performing an inverse orthogonaltransformation process corresponding to the orthogonal transformationprocess of the image encoding device 1 to the transformation coefficientobtained from the inverse quantizer 51, and outputs the reconstructedprediction residual signal to the decoded image generator 55.

Under the control of the inverse orthogonal transformation selectioncontroller 60, the filter processor 54 operates in the same manner asthat illustrated and described with reference to FIG. 5. That is, thefilter process in the filter processor 54 according to the presentembodiment is performed when the predicted image is the neighboringpixel non-reference prediction, that is, in this example, when it is theinter-prediction, the pixel signal (prediction signal) of thepredetermined filter process target region is selected from thepredicted image, a predicted image including a new prediction signal isgenerated by performing a low-pass filter process on the pixel signal(prediction signal) of the selected filter process target region byusing decoded signals (decoded neighboring signals) neighboring on theleft side or the upper side of the predicted image, and is output to thedecoded image generator 55. For example, as illustrated in FIG. 5, underthe control of the inverse orthogonal transformation selectioncontroller 60, the filter processor 54 performs a smoothing filterprocess with a predetermined weighting factor for each of the horizontalfilter region, the vertical filter region, and the angular filter regionin the filter process target region Sf among the pixel signals(prediction signals) in the block Bkp of the predicted image by usingthe decoded neighboring signal So around the block boundary of the blockBkp of the predicted image, and outputs the resultant signal to thedecoded image generator 55.

The operation of the inverse orthogonal transformation selectioncontroller 60 will be described with reference to FIG. 18. FIG. 18 is aflowchart of the process of the inverse orthogonal transformationselection controller 60 according to an example in the image decodingdevice 5 of the present embodiment. First, the inverse orthogonaltransformation selection controller 60 grasps a current processing stateby receiving, from the predicted image selector 59, a notification as towhether the current processing state is a neighboring pixelnon-reference prediction (step S21).

Subsequently, when the current processing state is the neighboring pixelnon-reference prediction (step S21: Yes), the inverse orthogonaltransformation selection controller 60 determines whether the orthogonaltransformation process (for example, DST) in which the end of thetransformation base is closed is used with respect to the transformationcoefficient, with reference to the transformation type identificationsignal obtained from the image encoding device 1, and whether theorthogonal transformation process (for example, DCT) in which the end ofthe transformation base is opened is used with respect to thetransformation coefficient (step S22).

Subsequently, when the inverse orthogonal transformation selectioncontroller 60 determines that the orthogonal transformation process (forexample, DST) in which the end of the transformation base is closed withrespect to the transformation coefficient is used (step S23: Yes), theinverse orthogonal transformation selection controller 60 performsselective control by instructing the filter processor 54 to execute thelow-pass filter process and instructing the inverse orthogonaltransformer 52 to apply the inverse orthogonal transformation process(for example, IDST) in which the end of the transformation base isclosed with respect to the transformation coefficient reconstructedthrough the inverse quantizer 51, and causes the inverse orthogonaltransformer 52 to generate a prediction residual signal to be added tothe predicted image obtained through the filter processor 54 (step S24).

Subsequently, when the inverse orthogonal transformation selectioncontroller 60 determines that the orthogonal transformation process (forexample, DCT) in which the end of the transformation base is opened withrespect to the transformation coefficient is used (step S23: No), theinverse orthogonal transformation selection controller 60 instructs thefilter processor 54 not to execute the low-pass filter process, performsselective control by instructing the inverse orthogonal transformer 52to apply the inverse orthogonal transformation process (for example,IDCT) in which the end of the transformation base is opened with respectto the transformation coefficient reconstructed through the inversequantizer 51, and causes the inverse orthogonal transformer 52 togenerate a prediction residual signal to be added to the predicted imageobtained through the filter processor 54 (step S25).

At the time of the prediction other than the neighboring pixelnon-reference prediction such as the intra prediction (step S21: No),the inverse orthogonal transformation selection controller 60 controlsthe inverse orthogonal transformer 52 and the filter processor 54 sothat the presence or absence of the execution of the filter process andthe orthogonal transformation type are executed according to apredetermined method (step S26).

As described above, since the inverse orthogonal transformer 52 and thefilter processor 54 are controlled to determine a type of the inverseorthogonal transformation process to be applied and the presence orabsence of the execution of the filter process with reference to thetransformation type identification signal obtained from the imageencoding device 1, the inverse orthogonal transformation selectioncontroller 60 can reduce the residual component in the predictionresidual signal on the image encoding device 1 side and can performdecoding with a small amount of information, thereby improving theencoding efficiency by a series of processes on the encoding side andthe decoding side.

In particular, even at the time of the neighboring pixel non-referenceprediction, since the inverse orthogonal transformation process (forexample, IDST) in which the end of the transformation base is closed isused for the block of the prediction residual signal obtained based onthe block of the predicted image after the filter process, withcorresponding to the encoding device 1 side, not only the phase but alsothe feature of the transformation base corresponding to the end point ofthe prediction region are matched, thereby improving the encodingefficiency more remarkably.

According to the image encoding device 1 and the image decoding device 5of the present embodiment configured as described above, since a signalerror occurring between the prediction signal of the predicted image andthe neighboring decoded block signal is reduced and the encodingefficiency is improved, it is possible to realize an image encodingdevice and an image decoding device of a video encoding method with highencoding efficiency.

Fourth Embodiment (Image Encoding Device)

Next, the filter processor 12 related to a predicted image, which is amain component of the present disclosure in the image encoding device 1according to a fourth embodiment of the present disclosure, will bedescribed with reference to FIGS. 19 and 20, and the image encodingdevice 1 of a specific typical example will be described with referenceto FIGS. 4, 21, and 22. The following description focuses on differencesfrom the above-described embodiment. That is, in the present embodiment,the same reference numerals are assigned to the same components as thosein the above embodiment, and a further detailed description thereof willbe omitted.

Comprehensively, similarly to the intra prediction for a predicted imageused for the inter-prediction, the image encoding device 1 of thepresent embodiment reduces the prediction residual signal for theleftmost and uppermost regions of the predicted image of theinter-prediction by applying, for example, a low-pass filter process byusing an encoded and decoded neighboring signal, divides the block ofthe prediction residual signal, and applies one of the DST and the DCTaccording to the directionality of the low-pass filter process.According to the current H.265 standard, the block size of the predictedimage to which the DST can be applied is limited to only a small blocksize (for example, 4×4). Therefore, in the image encoding device 1 ofthe present embodiment, in the inter-prediction in the block of a largerpredicted image, after applying the low-pass filter process to thepredicted image, block division is performed up to a prescribed blocksize (for example, a block size to which DST according to the standardis allowed to be applied), and then which one of the DST and the DCT isapplied is determined without any flag and the corresponding orthogonaltransformation process is executed.

FIG. 19A is a block diagram of the periphery of the filter processor 12with respect to a predicted image in the image encoding device 1according to the present embodiment, and FIG. 19B is an explanatorydiagram illustrating an example of the filter process of the predictedimage in the filter processor 12.

As illustrated in FIG. 19A, the image encoding device 1 according to thepresent embodiment includes the neighboring pixel non-referencepredictor 11, the filter processor 12, the prediction residual signalgenerator 13, and the orthogonal transformer 14. The orthogonaltransformer 14 includes a block divider 141 and an orthogonaltransformation selection applier 142.

The filter processor 12 in the present embodiment is a functional partthat generates a prediction signal including a new prediction signal byperforming a low-pass filter process on the prediction signal by usingdecoded signals (decoded neighboring signals) neighboring to the leftside and the upper side of the predicted image, among the pixel signals(prediction signals) in the block of the predicted image generated bythe neighboring pixel non-reference predictor 11, and outputs thepredicted image to the prediction residual signal generator 13.

For example, as illustrated in FIG. 19B, when the block Bkp of thepredicted image including 8×8 pixel signals p corresponding to the blockBko of the encoding target region of the original image is generated bythe neighboring pixel non-reference predictor 11, the filter processor12 generates a predicted image including a new prediction signal byperforming a low-pass filter process such as a smoothing filter(indicated by a double arrow) such as a horizontal filter, a verticalfilter, and an angular filter by setting pixel signals (predictionsignals) on the leftmost side and the uppermost side in the predictedimage as a signal Sf of the filter target region, by using a decodedneighboring signal So in the vicinity of the block boundary of the blockBkp of the predicted image neighboring on the predicted image.

More specifically, in the example illustrated in FIG. 19B, for thesmoothing filter target pixel on the upper side of the block Bkp of thepredicted image, a smoothing filter process is performed by using thedecoded neighboring signal (pixel signal of a locally decoded image)located at the same coordinates in the transverse direction, and for thesmoothing filter target pixel on the left side, a smoothing filterprocess is performed by using the decoded neighboring signal (pixelsignal of the locally decoded image) located at the same coordinate inthe longitudinal direction. For the target pixels located on the upperside and the left side, a smoothing filter process is performed by usingthe pixels of the locally decoded video on the upper and left sides. Inthe example illustrated in FIG. 19B, a 3-tap smoothing filter processusing two pixels of the reference pixel is applied. For example, alow-pass filter process such as ¼ [1 2 1] is applied. The number of tapsand reference pixels in the low-pass filter process are not limited tothis example.

The orthogonal transformer 14 includes the block divider 141 and theorthogonal transformation selection applier 142. The block divider 141divides the block of the prediction residual signal input from theprediction residual signal generator 13 into a block shape designated inadvance and outputs the same to the orthogonal transformation selectionapplier 142. For each block of the block-divided prediction residualsignal, the orthogonal transformation selection applier 142 selectivelyapplies a plurality of types of orthogonal transformation processes (acombination of longitudinal/transverse DST and longitudinal/transverseDCT) according to the position to which the filter process is applied bythe filter processor 12.

For example, as illustrated in FIG. 19B, the block divider 141 dividesthe block of the prediction residual signal with respect to the blockBkp of the predicted image including a 8×8 pixel signal p into fourgroups of 4×4 prediction residual signals (upper left, upper right,lower left, and lower right) and outputs the divided blocks to theorthogonal transformation selection applier 142. At this time, theorthogonal transformation selection applier 142 applies a plurality oftypes of orthogonal transformation processes (a combination of thelongitudinal/transverse DST and the longitudinal/transverse DCT)corresponding to the positions to which the smoothing filter process isapplied, in order to be able to handle transformation coefficients usingthe high degree of correlation for blocks of decoded neighboring signalswith respect to the divided block of the prediction residual signallocated at the upper end and the left end including the pixel positionto which the smoothing filter process is applied among the dividedblocks divided by the block divider 141, and applies the orthogonaltransformation process of longitudinal and transverse DCTs for the otherdivided blocks. The longitudinal direction means the vertical direction,and the transverse direction means the horizontal direction. Here, it isassumed that the orthogonal transformation process including thelongitudinal DST and the transverse DCT is a first orthogonaltransformation process, the orthogonal transformation process includingthe transverse DST and the longitudinal DCT is a second orthogonaltransformation process, the orthogonal transformation process includingthe longitudinal and transverse DSTs is a third orthogonaltransformation process, and the orthogonal transformation processincluding the longitudinal and transverse DCTs is a fourth orthogonaltransformation process.

In the example illustrated in FIG. 19B, an example of dividing the blockof the prediction residual signal for the block Bkp of the predictedimage including the 8×8 pixel signal p into four blocks (upper left,upper right, lower left, and lower right) of the 4×4 prediction residualsignal is illustrated, but the divided blocks (divided blocks located atthe upper end and the left end in the present example) at positionsneighboring to the decoded neighboring signal in the block of theprediction residual signal corresponding to the block of the predictedimage having a block size larger than a size (8×8 in the presentexample), which is doubled in the longitudinal and transverse directionswith respect to a predetermined specified block size (4×4 in thisexample), are assumed that the block size is specified (4×4 in thisexample), and the other blocks are divided so that the block size is aslarge as possible, and it is preferable to perform re-divisionappropriately according to the feature of the image to be encoded. Forexample, depending on the characteristics of the image to be encoded, asillustrated in FIG. 20A, the divided blocks located at the upper end andthe left end of the block of the prediction residual signal of the blocksize of 32×32 is 4×4, and it can be divided so as to become a dividedblock of a block size enlarged as it goes away from the upper end andthe left end. By performing the block division in this manner, theorthogonal transformation selection applier 142 in the latter stage canselect and apply the orthogonal transformation suitable for the signalcharacteristics with respect to the divided blocks of the predictionresidual signal including the smoothing filter process target pixelposition.

For example, as illustrated in FIG. 20A, if the pixel to which thesmoothing filter process is applied is on the uppermost side (that is,if the application direction of the smoothing filter process is thevertical direction) among the divided blocks of the divided predictionresidual signal, the orthogonal transformation selection applier 142applies the first orthogonal transformation process in the longitudinalDST and the transverse DCT to the divided block Bkt1 of the predictionresidual signal located on the uppermost side (see FIG. 20B), and if thepixel to which the smoothing filter process is applied is on theleftmost side (that is, if the application direction of the smoothingfilter process is the horizontal direction), the orthogonaltransformation selection applier 142 applies the second orthogonaltransformation process in the transverse DST and the longitudinal DCT tothe divided block Bkt2 of the prediction residual signal located on theleftmost side (see FIG. 20C). Then, if the pixel to which the smoothingfilter process is applied is the uppermost side and is the leftmost side(that is, if the application direction of the smoothing filter processis the application direction of the angular filter process), the thirdorthogonal transformation process in the longitudinal and transverseDSTs is applied to the divided block Bkt3 located in the corner region.In addition, for the divided block Bkt4 of the prediction residualsignal having no pixel to which the smoothing filter process is applied,the fourth orthogonal transformation process of DCTs is applied in thevertical and transverse directions. Even in the case of the blockdivision as illustrated in FIG. 19B, the orthogonal transformationprocess is similarly performed. This makes it possible to handletransformation coefficients that maximize the high correlation of theblocks of the decoded neighboring signals.

Next, the inter predictor 11 a based on the inter-prediction will bedescribed as a representative example of the neighboring pixelnon-reference predictor 11, and a configuration and an operation exampleof the image encoding device 1 that performs the filter process on theprediction signal of the block of the predicted image by the filterprocessor 12 will be described with reference to FIGS. 21 to 23.

FIG. 21 is a block diagram illustrating an example of the image encodingdevice 1 according to the present embodiment. The image encoding device1 illustrated in FIG. 21 includes the pre-processor 10, the interpredictor 11 a, the filter processor 12, the prediction residual signalgenerator 13, the orthogonal transformer 14, the quantizer 15, theinverse quantizer 16, the inverse orthogonal transformer 17, the decodedimage generator 18, the in-loop filter 19, the frame memory 20, theintra predictor 21, the motion vector calculator 22, the entropy encoder23, and the predicted image selector 24.

The orthogonal transformer 14 performs a predetermined orthogonaltransformation process on the prediction residual signal input from theprediction residual signal generator 13, and generates a signal of atransformation coefficient. The orthogonal transformer 14 includes theblock divider 141 and the orthogonal transformation selection applier142. The block divider 141 divides the block of the prediction residualsignal input from the prediction residual signal generator 13 into ablock shape designated in advance and outputs the same to the orthogonaltransformation selection applier 142. For each divided block of theblock-divided prediction residual signal, the orthogonal transformationselection applier 142 applies a plurality of types of orthogonaltransformation processes (a combination of longitudinal/horizontal DSTand horizontal/vertical DCT) according to the position to which thefilter process is applied by the filter processor 12. The block divisionparameter related to these block division process is output to theentropy encoder 23 (and the inverse orthogonal transformer 17) as one ofthe encoding parameters to be used as additional information of theencoding process.

The inverse orthogonal transformer 17 reconstructs the predictionresidual signal by performing an inverse orthogonal transformationprocess on the inversely quantized transformation coefficient signalobtained from the inverse quantizer 16 and outputs the reconstructedprediction residual signal to the decoded image generator 18. Inparticular, the inverse orthogonal transformer 17 determines the type ofthe orthogonal transformation process for the transformationcoefficients that have been divided into blocks, as in FIG. 20, performsthe inverse orthogonal transformation process corresponding to eachorthogonal transformation process, combines each divided block of theprediction residual signal obtained for parallel process to reconfigurea block corresponding to the block size of the predicted image, andoutputs the resultant signal to the decoded image generator 18.

An operation example of the filter processor 12 will be described withreference to FIG. 22. As illustrated in FIG. 22, when the predictedimage from the intra predictor 21 or the inter predictor 11 a is input(step S1), the filter processor 12 identifies whether the predictedimage is the neighboring pixel non-reference prediction, that is, inthis example, identifies whether the predicted image is theinter-prediction based on the signal selection by the predicted imageselector 24 (step S2). The same applies to a case where the neighboringpixel non-reference prediction is an intra block copy prediction or across component signal prediction to be described above.

Subsequently, when the filter processor 12 determines that the predictedimage is not the neighboring pixel non-reference prediction, that is,that the predicted image is the intra prediction in this example (stepS2: No), the filter processor 12 determines whether or not to performthe filter process by a predetermined processing method (step S5). Whenthe filter process is not performed on the predicted image of the intraprediction (step S5: No), the filter processor 12 outputs the predictionsignal to the prediction residual signal generator 13 without performingthe filter process thereon (step S6). On the other hand, when filterprocess is performed on the predicted image of the intra prediction(step S5: Yes), the filter processor 12 proceeds to step S3. Since thefilter process on the predicted image of the intra prediction may beperformed in the same manner as the currently defined H.265 and is notdirectly related to the gist of the present disclosure, the descriptionof the step S3 is described as an example that performs the filterprocess on the predicted image of the inter-prediction.

When the filter processor 12 determines that the predicted image is theneighboring pixel non-reference prediction, that is, theinter-prediction in this example (step S2: Yes), the filter processor 12extracts a pixel signal of a filter process target region in advancefrom the predicted image (prediction signal) (step S3). For example, asillustrated in FIG. 19B, the pixel signals (prediction signals) on theleftmost side and the uppermost side of the predicted image among thepixel signals (prediction signals) in the 8×8 predicted image block aredefined as a filter process target regions Sf.

Subsequently, the filter processor 12 generates a predicted imageincluding a new prediction signal by performing a low-pass filterprocess on the pixel signal (prediction signal) of the selected filterprocess target region by using the decoded signal (the decodedneighboring signal) neighboring on the left side and the upper side ofthe predicted image, and outputs the generated predicted image to theprediction residual signal generator 13 (step S4).

It should be noted that the filter process by the filter processor 12may be a low-pass filter process, filter processes other than thesmoothing filter may be applied, and the region of the decodedneighboring signal or the weighting factor used for the filter processare not necessarily limited to the example illustrated in FIG. 19B. Inaddition, the filter type, the filter process target region, and theweighting coefficient, which are related to the filter process performedby the filter processor 12, may be configured to be transmitted asadditional information of the encoding process, but need not betransmitted if they are determined in advance between transmission andreception (between encoding and decoding).

By performing the filter process on the predicted image by the filterprocessor 12 in this way, a signal error occurring between the pixelsignals (prediction signals) on the leftmost side and the uppermost sidein the predicted image and the neighboring decoded signal is reduced,regardless of the type of the orthogonal transformation process of theorthogonal transformer 14, and it is possible to improve the encodingefficiency of the video.

Next, an operation example of the orthogonal transformer 14 will bedescribed with reference to FIG. 23. As illustrated in FIG. 23, theorthogonal transformer 14 divides the block of the prediction residualsignal input from the prediction residual signal generator 13 into blockshapes designated in advance by the block divider 141 (step S11). Atthis time, the block divider 141 sets the divided block of theprediction residual signal located at the upper end and the left end toa predetermined block size (4×4 in this example) for the block of thepredicted image of the block size larger than 8×8 (for example, for thepredicted image of the block size of 32×32), divides the other blocksinto so that the block size is as large as possible, and appropriatelyperforms re-division according to the feature of the image to beencoded.

Subsequently, when the orthogonal transformation process is performed onthe divided blocks of the divided prediction residual signal by theorthogonal transformation selection applier 142, the orthogonaltransformer 14 determines whether the pixel position to which the filterprocess by the filter processor 12 is applied is included (that is,whether it is the prediction residual signal of the block including theuppermost or leftmost pixel position of the predicted image) (step S12).When the orthogonal transformer 14 determines that it is not the dividedblock including the pixel position to which the filter process isapplied (step S12: No), the orthogonal transformer 14 applies the fourthorthogonal transformation process of the DCT to the divided blocks ofthe prediction residual signal in the longitudinal and transversedirections (step S15).

On the other hand, when the orthogonal transformer 14 determines that itis the divided block including the pixel position to which the filterprocess by the filter processor 12 is applied (step S12: Yes), it isdetermined whether only the filter process in the vertical direction bythe filter processor 12 has been applied (that is, whether it is theprediction residual signal of the block including only the uppermostpixel of the predicted image) (step S13), and when only the filterprocess in the vertical direction is applied (step S13: Yes), the firstorthogonal transformation process of the longitudinal DST and thetransverse DCT is applied to the divided blocks of the predictionresidual signal (step S16).

Subsequently, when only the filter process in the vertical direction isnot applied (step S12: No), it is determined whether only the filterprocess in the horizontal direction by the filter processor 12 has beenapplied (that is, whether it is the prediction residual signal of theblock including only the leftmost pixel of the predicted image) (stepS14), and when only the filter process in the horizontal direction isapplied (step S14: Yes), the second orthogonal transformation process ofthe transverse DST and the longitudinal DCT is applied to the dividedblocks of the prediction residual signal (step S17).

When it is determined that the horizontal and vertical filter processesby the filter processor 12 are applied (that is, when it is theprediction residual signal of the block including the uppermost andleftmost pixel positions of the predicted image) (step S14: No), thethird orthogonal transformation process of the longitudinal andtransverse DSTs is applied to the divided block of the predictionresidual signal (step S18).

In FIG. 23, it has been described that the orthogonal transformationprocess is sequentially performed in order to enhance the understandingof the processing contents, but parallel processing can be performed onthe implementation. This makes it possible to handle transformationcoefficients that maximize the high correlation of the blocks of thedecoded neighboring signals.

(Image Decoding Device)

Next, a filter processor 54 and its peripheral functional blocks, whichare the main components of the image decoding device 5 according to thepresent embodiment of the present disclosure, will be described withreference to FIG. 24, and a specific typical example of the imagedecoding device 5 will be described with reference to FIG. 25.

FIG. 24 is a block diagram of the periphery of the filter processor 54with respect to the predicted image in the image decoding device 5according to the present embodiment. As illustrated in FIG. 24, theimage decoding device 5 according to the present embodiment includes aninverse orthogonal transformer 52, a neighboring pixel non-referencepredictor 53, a filter processor 54, and a decoded image generator 55.The inverse orthogonal transformer 52 includes an inverse orthogonaltransformation selection applier 521 and a block reconfigurer 522.

The inverse orthogonal transformer 52 receives the stream signaltransmitted from the image encoding device 1 side, performs an inverseorthogonal transformation process on transformation coefficientsreconstructed through a entropy decoding process and an inversequantization process, and outputs the obtained prediction residualsignal to the decoded image generator 55. In particular, the inverseorthogonal transformer 52 includes an inverse orthogonal transformationselection applier 521 and a block reconfigurer 522. The inverseorthogonal transformation selection applier 521 determines the type ofthe orthogonal transformation process as in FIG. 23 for thetransformation coefficients block-divided by the image encoding device 1side, performs the inverse orthogonal transformation processcorresponding to the orthogonal transformation process in the imageencoding device 1, and outputs the divided block of the predictionresidual signal to the block reconfigurer 522. The block reconfigurer522 reconstructs a block corresponding to the block size of thepredicted image by connecting the divided blocks of the predictionresidual signal obtained for parallel processing, and outputs the blockto the decoded image generator 55.

Next, the inter predictor 53 a based on the inter-prediction will bedescribed as a representative example of the neighboring pixelnon-reference predictor 53, and a configuration and an operation exampleof the image decoding device 5 that performs the filter process on theprediction signal based on the inter-prediction by the filter processor54 will be described with reference to FIG. 25.

FIG. 25 is a block diagram illustrating an example of the image decodingdevice 5 according to the present embodiment. The image decoding device5 illustrated in FIG. 25 includes an entropy decoder 50, an inversequantizer 51, an inverse orthogonal transformer 52, an inter predictor53 a, a filter processor 54, a decoded image generator 55, an in-loopfilter 56, a frame memory 57, an intra predictor 58, and a predictedimage selector 59.

The inverse orthogonal transformer 52 reconstructs the predictionresidual signal by performing an inverse orthogonal transformationprocess corresponding to the orthogonal transformation process of theimage encoding device 1 to the transformation coefficient obtained fromthe inverse quantizer 51, and outputs the reconstructed predictionresidual signal to the decoded image generator 55. In particular, theinverse orthogonal transformer 52 includes an inverse orthogonaltransformation selection applier 521 and a block reconfigurer 522. Theinverse orthogonal transformation selection applier 521 performs theinverse orthogonal transformation process to the transformationcoefficients block-divided by the image encoding device 1, and outputsthe divided block of the prediction residual signal to the blockreconfigurer 522. The block reconfigurer 522 constructs a blockcorresponding to the block size of the predicted image by connecting thedivided blocks of the prediction residual signal obtained for parallelprocessing, and outputs the block to the decoded image generator 55.

The filter processor 54 operates in the same manner as that illustratedand described with reference to FIG. 22. That is, the filter processaccording to the present disclosure in the filter processor 54 isperformed when the predicted image is the neighboring pixelnon-reference prediction, that is, in this example, when it is theinter-prediction, the pixel signal (prediction signal) of thepredetermined filter process target region is selected from thepredicted image, a predicted image including a new prediction signal isgenerated by performing a low-pass filter process on the pixel signal(prediction signal) of the selected filter process target region byusing decoded signals (decoded neighboring signals) neighboring on theleft side or the upper side of the predicted image, and is output to thedecoded image generator 55. For example, as illustrated in FIG. 19B, thefilter processor 54 performs a smoothing filter process with apredetermined weighting factor for each of the horizontal filter region,the vertical filter region, and the angular filter region in the filterprocess target region Sf among the pixel signals (prediction signals) inthe block Bkp of the predicted image by using the decoded neighboringsignal So around the block boundary of the block Bkp of the predictedimage, and outputs the resultant signal to the decoded image generator55.

According to the image encoding device 1 and the image decoding device 5of the present embodiment configured as described above, since a signalerror occurring between the prediction signal of the predicted image andthe neighboring decoded block signal is reduced and the encodingefficiency is improved, it is possible to realize an image encodingdevice and an image decoding device of a video encoding method with highencoding efficiency. That is, the residual component in the predictionresidual signal can be made smaller, and the encoding efficiency can beimproved by reducing the amount of information to be encoded andtransmitted.

Fifth Embodiment

Next, the periphery of filter processors 12 and 54 in the image encodingdevice 1 and the image decoding device 5 according to a fifth embodimentof the present disclosure will be described with reference to FIG. 26.In the filter processors 12 and 54 of the fourth embodiment, an examplethat generates the predicted image including the new prediction signalby performing the low-pass filter process on the prediction signal byusing only the decoded signals (decoded neighboring signals) neighboringto the left side and the upper side of the predicted image, among thepixel signals (prediction signals) in the block of the predicted imagegenerated by the neighboring pixel non-reference prediction has beendescribed.

On the other hand, the filter processors 12 and 54 of the presentembodiment acquire a locally decoded image for each divided block basedon the block division process of the block divider 141, replaces thepixel signal (prediction signal) in the block of the predicted imagegenerated by the neighboring pixel non-reference prediction whenever thelocally decoded image is acquired, and performs a low-pass filterprocess according to each block size by using the decoded signal(decoded neighboring signal) neighboring on the left side and the upperside of the predicted image among pixel signals (prediction signals) inthe replaced predicted image block and the locally decoded image foreach divided block in the block size unit of the divided block of theprediction residual signal.

Therefore, as illustrated in FIGS. 26A and 26B, the filter processors 12and 54 acquire a locally decoded image for each divided block based onthe block division process of the block divider 141, update thepredicted image by replacing the pixel signal (prediction signal) in theblock of the predicted image generated by the neighboring pixelnon-reference prediction whenever the locally decoded image is acquired,and perform a low-pass filter process according to each block size byusing the decoded signal (decoded neighboring signal) neighboring to theleft side and the upper side of the updated predicted image and thelocally decoded image of each divided block. That is, the filterprocessors 12 and 54 are different from the configurations illustratedin FIGS. 19A and 24, but the other components operate in the samemanner.

As an example of the operation, unlike the configuration illustrated inFIG. 24, the block reconfigurer 522 in the inverse orthogonaltransformer 52 illustrated in FIG. 26B complements the region other thanthe divided block to be processed with dummy data when reconfiguring theblock size corresponding to the predicted image by determining theposition of the divided block to be processed. This makes it possible toobtain a locally decoded image for each divided block. Alternatively,another loop may be configured so that a locally decoded image for eachdivided block can be obtained. Therefore, the inverse orthogonaltransformer 17 in the image encoding device 1 of the present embodimentmay be configured similarly to the inverse orthogonal transformer 52 inthe present embodiment. It should be noted that the same applies to thefourth embodiment, but the frame memory 20 can hold the locally decodedimage of each divided block so as to be used as a reference signal.

FIG. 27 is a flowchart of the filter processes 12 and 54 of thepredicted image in the image encoding device 1 or the image decodingdevice 5 according to the present embodiment. In addition, FIG. 28 is anexplanatory diagram of the filter processes 12 and 54 of the predictedimage in the image encoding device 1 or the image decoding device 5according to the present embodiment.

First, when the predicted image is input, the filter processors 12 and54 acquire a locally decoded image for each divided block based on theblock division process of the block divider 141, replace the locallydecoded image with a predicted image, and generate an updated predictedimage (step S21). For example, as illustrated in FIG. 28, the block Bkpof the 8×8 predicted image is sequentially replaced with the block BkLof the locally decoded image of the block size divided by four in theorder indicated by outlined arrows and updated, for example, but thefollowing process indicates the operation when the predicted image isupdated with the locally decoded image for each one of the dividedblocks (for example, “STEP 1” illustrated in the drawing).

Subsequently, the filter processors 12 and 54 determine whether thepredicted image is the neighboring pixel non-reference prediction, forexample, whether it is the inter-prediction (step S22). The same appliesto a case where the neighboring pixel non-reference prediction is anintra block copy prediction or a cross component signal prediction to bedescribed above.

Subsequently, when the filter processors 12 and 54 determine that thepredicted image is not the neighboring pixel non-reference prediction,for example, that the predicted image is the intra prediction in thisexample (step S22: No), the filter processors 12 and 54 determinewhether or not to perform the filter process by a predeterminedprocessing method (step S25). When the filter process is not performedon the predicted image of the intra prediction (step S25: No), thefilter processors 12 and 54 output the prediction signal to theprediction residual signal generator 13 or the decoded image generator55 without performing the filter process thereon (step S26). On theother hand, when filter process is performed on the predicted image ofthe intra prediction (step S25: Yes), the filter processors 12 and 54proceed to step S23. Since the filter process on the predicted image ofthe intra prediction may be performed in the same manner as thecurrently defined H.265 and is not directly related to the gist of thepresent disclosure, the description of the step S23 is described as anexample that performs the filter process on the predicted image of theinter-prediction.

When the filter processors 12 and 54 determine that the predicted imageis the neighboring pixel non-reference prediction, that is, theinter-prediction in this example (step S22: Yes), the filter processors12 and 54 extracts a pixel signal of a filter process target region inadvance from the predicted image (prediction signal) (step S23). Forexample, as illustrated in FIG. 28, the pixel signals (predictionsignals) on the leftmost side and the uppermost side with respect to theneighboring signal are determined as the filter process target region Sfin units of 4×4 divided blocks.

Subsequently, the filter processors 12 and 54 generate a predictedimages including a new prediction signal by performing a low-pass filterprocess on the pixel signal (prediction signal) of the selected filterprocess target region by using the decoded signal (decoded neighboringsignal) neighboring to the left side and/or the upper side for eachdivided block for the predicted image replaced by the locally decodedimage of the divided block and the locally decoded image, and outputsthe generated predicted image to the prediction residual signalgenerator 13 and the decoded image generator 55 (step S24).

In this manner, by performing the low-pass filter process by using thedecoded signal (decoded neighboring signal) neighboring to the left sideand/or the upper side of the predicted image and the locally decodedimage neighboring to the left side and/or the upper side for eachdivided block, for example, the pixel signal of the block Bkp of the 8×8predicted image is smoothed not only at the upper side and the left sidebut also at the boundary position of the divided block which is thecross-shape in this example (for example, “STEP 2” in the drawing), asillustrated in FIG. 28.

By configuring the filter processors 12 and 54 as in the presentembodiment and applying the filter processors 12 and 54 to the imageencoding device 1 and the image decoding device 5, respectively, it ispossible to improve the encoding efficiency and suppress thedeterioration of the image quality caused by this. In particular, evenin the signal of the divided block other than the divided block locatedon the leftmost side or the uppermost side of the predicted image whichcan be a prediction residual signal of a large value (the block locatedinside the predicted image when dividing the block), it is possible toreduce the values of the leftmost and the uppermost prediction residualsignals for each divided block.

In the example of the present embodiment, since all the divided blocksinclude pixel positions as the filter process targets, DST in thelongitudinal direction or the transverse direction is always applied.Therefore, in the example of the present embodiment, as in the fourthembodiment, as for the block shape upon block division, the dividedblock located at the upper end and the left end in the block of thecorresponding prediction residual signal is set to the predeterminedblock size (4×4 in this example), and it is preferable to performre-division appropriately according to the feature of the image to beencoded after the other blocks are divided so that the block size is aslarge as possible. It is unnecessary to divide the block size so as tohave an enlarged block size as it goes away from the upper end and theleft end thereof, all are divided up to a prescribed size (4×4 in thisexample) that can be applied with DST, and DST can be applied accordingto the processing order as illustrated in FIG. 23.

In addition, when comparing the fourth embodiment with the presentembodiment, in the fourth embodiment, since each divided block of theprediction residual signal can be processed independently, there is anadvantage that parallel processing is possible. In the presentembodiment, since locally decoded images in divided block units can beused sequentially, it is expected that the encoding efficiency can befurther improved.

In addition, since the block shape of the block division can be variablyset for each encoding target block and the block division parameter canbe transmitted as one of the encoding parameters, the fourth embodimentand the present embodiment may be combined.

The image encoding device 1 and the image decoding device 5 of eachembodiment can function as a computer, and a program for realizing therespective components according to the present disclosure can be storedin a memory installed internally or externally to the computer. Undercontrol of a central processing unit (CPU) or the like provided in thecomputer, a program in which processing contents for realizing thefunction of each component are written can be read from a memory asappropriate, and the functions of the respective components of the imageencoding device 1 and the image decoding device 5 of each embodiment canbe realized by the computer. Here, the function of each component may berealized as a part of hardware.

Although the present disclosure has been described by way of examples ofspecific embodiments, the present disclosure is not limited to theabove-described examples, and various modifications can be made withoutdeparting from the technical idea thereof. For example, in the examplesof the above-described embodiments, the example in which the block sizesof the encoding target region, the predicted image, and the orthogonaltransformation process target are the same has been described, but thesame applies to a case where the block size of the predicted image issmaller than the encoding target region, or a case where the block sizeof the orthogonal transformation process target is smaller than thepredicted image. In addition, the same as the orthogonal transformationprocess may be used according to use, or another orthogonaltransformation process may be used.

In addition, in the example of each above-described embodiment, theexample in which the image decoding device 5 decodes the transformationcoefficients related to the prediction residual signal encoded based onthe predicted image subjected to the filter process by the correspondingimage encoding device 1 as a target has been described, the imagedecoding device 5 according to the present disclosure can also decode asignal encoded without a filter process by the same process in theapplications where the decompression accuracy of the block end for theoriginal image is not considered.

INDUSTRIAL APPLICABILITY

According to the present disclosure, it is possible to realize an imageencoding device and an image decoding device using a video encodingmethod with high encoding efficiency, it is useful for improving theencoding efficiency of the video transmission.

What is claimed is:
 1. An image encoding device for encoding byblock-dividing an original image of a frame unit constituting a movingimage, the image encoding device comprising: a neighboring pixelnon-reference predictor configured to generate a predicted image of ablock unit including a prediction signal by a predetermined neighboringpixel non-reference prediction that performs a signal prediction foreach pixel signal of an original image of the block unit without using adecoded neighboring signal; a filter processor configured to controlexecution and non-execution of a low-pass filter process under apredetermined selection control, and at the time of the execution,generating a block of a new predicted image by performing the low-passfilter process on a prediction signal located at a boundary of a blockof the predicted image by using the decoded neighboring signalneighboring to the block of the predicted image; a prediction residualsignal generator configured to calculate an error of each predictionsignal of the block of the predicted image generated by the filterprocessor for each pixel signal of an encoding target block of theoriginal image and generating a prediction residual signal of the blockunit; an orthogonal transformer configured to selectively control one ofa plurality of types of orthogonal transformation processes under thepredetermined selection control, and performing an orthogonaltransformation process on the prediction residual signal of the blockunit to generate a transformation coefficient signal by the selectivelycontrolled orthogonal transformation process; and an orthogonaltransformation selection controller configured to selectively controlone of a first combination for applying a first orthogonaltransformation process in which an end of a transformation base isclosed among the plurality of types of orthogonal transformationprocesses according to the execution of the low-pass filter process anda second combination for applying a second orthogonal transformationprocess in which the end of the transformation base is opened among theplurality of types of orthogonal transformation processes according tothe non-execution of the low-pass filter process, based on apredetermined evaluation criterion at the time of the neighboring pixelnon-reference prediction as the predetermined selection control, andgenerating, as an encoding parameter, a transformation typeidentification signal indicating a type of the selected orthogonaltransformation process.
 2. The image encoding device according to claim1, wherein the predetermined neighboring pixel non-reference predictioncomprises any one of an inter-prediction, an intra block copyprediction, and a cross component signal prediction.
 3. The imageencoding device according to claim 1, wherein the orthogonaltransformation selection controller comprises a selector configured toselect the first combination when an effect of the low-pass filterprocess is equal to or higher than a predetermined level or excellent asan RD cost through the execution of the low-pass filter process, as thepredetermined evaluation criterion.
 4. The image encoding deviceaccording to claim 1, wherein the filter processor configured to performthe low-pass filter process such that a prediction signal neighboring tothe decoded neighboring signal is smoothed among prediction signals inthe block of the predicted image generated by the neighboring pixelnon-reference predictor.
 5. An image decoding device for decoding asignal encoded by block-dividing a frame constituting a moving image,the image decoding device comprising: an inverse orthogonaltransformation selection controller configured to acquire atransformation type identification signal indicating a type of anorthogonal transformation process selected on an encoding side among aplurality of types of orthogonal transformation processes, andselectively determining a type of corresponding inverse orthogonaltransformation process based on the transformation type identificationsignal at the time of a predetermined neighboring pixel non-referenceprediction that performs a signal prediction without using a decodedneighboring signal for each pixel signal of a block unit; an inversequantizer configured to reconstruct a transformation coefficient signalby performing a corresponding inverse quantization process on atransformation coefficient quantized by a predetermined quantizationprocess as a signal encoded by block-dividing a frame constituting amoving image; an inverse orthogonal transformer configured toreconstruct a prediction residual signal of the block unit by performingthe inverse orthogonal transformation process determined based on thetransformation type identification signal with respect to thereconstructed transformation coefficient signal; a neighboring pixelnon-reference predictor configured to generate a predicted image of theblock unit including a prediction signal by the neighboring pixelnon-reference prediction; and a filter processor configured to generatea block of a new predicted image by performing a low-pass filter processon a prediction signal located at a boundary of a block of the predictedimage by using the decoded neighboring signal neighboring to the blockof the predicted image when a first orthogonal transformation process inwhich an end of a transformation base is closed by the transformationtype identification signal.
 6. The image decoding device according toclaim 5, wherein the inverse orthogonal transformation selectioncontroller comprises acquirer configured to acquire the transformationtype identification signal as an encoding parameter indicating that oneof the first orthogonal transformation process in which the end of thetransformation base is closed and a second orthogonal transformationprocess in which the end of the transformation base is opened among theplurality of types of orthogonal transformation processes is applied tothe encoding side.
 7. A program for causing a computer to function asthe image encoding device according to claim
 1. 8. A program for causinga computer to function as the image decoding device according to claim5.